DataJoint Adds Native Support for CWL Pipeline Conversion

DataJoint has announced native support for converting Common Workflow Language (CWL) pipelines into its platform, according to a press release. The feature enables research organizations to migrate existing CWL workflows into DataJoint’s governed, provenance-rich infrastructure without redeveloping them.

The new conversion layer reads CWL definitions and executes them as native DataJoint pipelines. It allows researchers to extend workflows using DataJoint’s Python-based schema framework and supports interpreted execution, with compiled execution planned for future updates.

Key capabilities include automatic provenance tracking for every step, granular error recovery that allows individual task retries, and real-time workflow state querying. The system also decomposes pipelines into parallelizable steps that can pause and resume without data loss.

As part of the conversion, DataJoint builds a structured database around the scientific entities produced by each workflow. This approach creates a queryable record of inputs, outputs, and relationships, turning computational workflows into traceable scientific datasets suitable for AI-driven research.