Hacker News Reader

Comment by jdeaton | Hacker News Reader

One of the biggest issues I've observed with workflow systems is an inability to get return values from each subprocess and then decide based on those which subprocesses to instantiate next. It always seems like the workflow execution graph is static in my experience. Is that correct?

View on HN

At a prior company we just wrote our own custom workflow engine which was basically a DAG processor where each node called some code. Of course, the code called could modify the DAG in place and accomplish exactly what you're describing. It worked quite well but we didn't polish it as much as we could have. I've been wanting to do a clean-room reimplementation as of late just because I miss certain things about it.

Not sure what you've tried, but Temporal (which I think is the most mature and well-designed open source workflow engine available right now) supports using the result of a "subprocess" to decide the next steps. The only requirement is that your logic is consistent across workflow runs.

That sounds very similar to AWS Step Functions to me. You define a workflow graph in JSON, then run the workflow and hand out subprocesses to lambdas etc.

I've definitely seen workflows written as single step graphs, where the output is opening a new workflow with a different single step

It was hard to read and understand

The couple workflow engines I've used supports that. It sounds crazy to not support something that obvious

Maybe before, but i feel that is not a problem anymore. Windmill.dev solves this with easee. Very good UI on top of the workflow engine.

Why would you want to rewrite the code of the workflow during execution, vs defining that logic upfront?

That's like asking why anyone would want to use an if statement. "Why would you want to change which instructions that get executed during the program? Just decide up front!". Because the decision cannot be made ahead of time.

Reply on news.ycombinator.com