Hacker News Reader

Comment by adolph | Hacker News Reader

Another day, another workflow DSL.

  * looks like yaml
  * has curly braces to look programmery
  * whitespace might be meaningful
  * has pipes like a bash script

https://www.commonwl.org/

https://github.com/common-workflow-language/common-workflow-...

mea culpa: The above was based on a first look at something titled "A DSL for parallel and and scalable computational pipelines"a as opposed to "Java workflow manager with Groovy language scripting." The presented screenshot still looks to me like an unholy union of yaml, js, py and sh. If that sounds groovy to you; have fun.

View on HN

In all fairness, they predate the competition (2013): https://github.com/nextflow-io/nextflow/releases?page=25

As GP referenced CWL, while NF had appeared first in terms of the bioinformatics world Nextflow, CWL, Snakelike, and WDL all erupted close enough to each other to be equal-ish. The people were aware of each other but they were all so nascent that it wasn't clear if it was worth joining in or not. At the end of the day these all came from groups trying to scratch particular itches, and not everyone agreed on the right way to scratch.

However all of them were rejections of prior models as well as the workflow solutions prominent in the business space.

Yeah, the thing that I find disappointing is that there is a lot of science value locked into the different systems of describing a workflow, pipeline or DAG. Like you said, they all had different itches to scratch and even some barebones "standards" like csv have flavors/extensions/etc.

They try to address similar solutions, but comparing snakemake and nextflow doesn't do either tool a favour. They use different computation models, nextflow is based on dataflow programming and therefore schedules processes dynamically as new data comes in, while snakemake is pull-based and schedules the processes based on the dag defined by the dependencies. Anyhow they are both great tools.

While true that’s a minor distinction when comparing the clusters of bioinformatics workflow systems vs workflow systems aimed at different domains

While these two are aimed at bioinformatics, they are general purpose enough that you can apply them to any computational workflow. I can say they saved my PhD

In fairness, this is an old problem with many other contenders. This issue is as old as batch schedulers. FWIW, I was at an ISMB conference in 2005 that had at least 2-3 workflow managers presented.

If you refer to nextflow,the syntax is basically groovy

Indeed, it's a fully functional scripting language that is an extension of groovy. So "looks programmery" is more than a little reductive.

Edit: functional as in, it's not half-hearted, not functional as in functional programming.

Interesting. From the docs:

The Nextflow scripting language is an extension of the Groovy programming language. Groovy is a powerful programming language for the Java virtual machine. The Nextflow syntax has been specialized to ease the writing of computational pipelines in a declarative manner.

https://www.nextflow.io/docs/latest/script.html?highlight=gr...

Its really just a layer on top that uses the groovy DSL features

http://docs.groovy-lang.org/docs/latest/html/documentation/c...

Reply on news.ycombinator.com