Comment by peter_l_downs | Hacker News Reader

Comment by peter_l_downs | Hacker News Reader

Looks like a great side project, congrats on launching. My basic feedback is that you should set expectations on the project page by answering the following questions:

- Why would someone use this instead of Airflow/Cadence/Temporal/Databuilderframework?

- What does this look like when it's used? Most frameworks provide some kind of example project, you should too.

- Related, but more specifically, what does the `IDataStore` interface contract mean? Beyond the two functions that I have to implement, are there any considerations related to the overall performance/scalability/durability of the system? Would it make sense to use a disk-backed store, or Redis, or Postgres?

- How do I observe the system? Which workflows are running, which have failed, what the current state is, etc. Are there metrics? Logs?

All of this is based on the assumption you want people to adopt this framework. If it's just a cool side project, that's fine too, but you should probably try to set that expectation in the README.

Super helpful feedback @peter. Thanks a ton!

I've noted all of these and I'll modify the README to include them. Thank you for taking the time to go through in such detail :)

Given that the current state of a workflow:

- is inherently invisible

- all we can really check in the DB is if, for a workflow, the available data contains the target data;

how would I address observability concerns?

This is a function of a lack of workflow states due to the lower levels of abstraction it operates on. User defined workflow states would do the trick, but I suppose that would take writing some more code after integrating the framework.

I would recommend either updating your framework to allow for instrumentation (logs/metrics/etc) or showing how to add that instrumentation in "user defined workflow states" via an example application.

The current pitch, which is "this is a workflow orchestration framework that does not allow for any monitoring or observability" is a complete non-starter. You may want to consider how other projects keep track of workflow state and allow for it to be instrumented.

Fair enough, will do. Thanks!

I don't generally believe in orchestrators (they miss the point, things are not single computers and neither is the world) and so I have that feedback here but also for:

> Airflow/Cadence/Temporal/Databuilderframework?

Which don't really think about modelling non-centralized things.

This of course doesn't mean they're not useful, it's just that they don't have what I believe is a good long-term value proposition.

I'm incredibly biased because I'm working on programmatic, real-time modelling of distributed systems with https://github.com/purpleidea/mgmt/

I don't think you understand what Temporal does.

Can you elaborate on what you mean by modelling non-centralized things?

Reply on news.ycombinator.com