Show HN: Odin – The Observable and Distributed Workflow System
github.com
github.com
1. the schedule syntax. It can seem much easier to work directly with English, but there's a reason most systems use cron-style schedules and it isn't because it's legacy. There's a benefit in having a syntax that is "alien" when you're crossing that human/computer/dev border, as it forces your users to think about how to achieve something. The other half of the reason is because it's a lot easier to validate a strict syntax with a formal grammar than anything even close to natural language. I can basically guarantee you that if someone had come up with a way to accurately parse english recurrence statements, it would have already been adopted by cron in the past (or at least a website like crontab guru[0].
2. There is no description of the job execution semantics in the distributed sense. Is it at most once, or at least once? This is important if you're designing jobs, so if someone is switching to your system from an existing system with a different semantic it will cause a lot of friction especially if you don't even list it in your documentation.
3. You have two data stores. You're using Raft for backing your key/value store, and mongo for both job metadata and also communicating with the dashboard. This is a bit confusing because in the docs you wrote that metadata goes in a sidecar yaml file. What job metadata are you referring to then? Results? Fundamentally, having multiple data stores makes your projects orders of magnitude more complicated to operate and monitor. My guess is you'll see this as a major pain point in the future when you want to scale a cluster in different ways (more jobs, more frequent jobs, longer histories, more nodes in the cluster).
All in all, it looks like a pretty decent distributed cron, but I wouldn't touch your project for my production systems. It's lacking a lot of features we need for getting Legacy (TM) work done[1] like running a specific job on a specific worker because it needs access to special hardware or some filesystem or because our PHB decided that as a policy only one of our servers can run billing code on Tuesdays.
[1]: "Legacy code is code that makes money"
1. I completely agree. We have recently acknowledged the power in an "alien" syntax as you put it. In v2.0.0 Odin will also support the classic cron schedule string syntax!
2. You are right, the repo is lacking a detailed description on this. The execution semantics boils down to at most once.
3. The Yaml file just lends some simple schedule and runtime info to the job. We actually would rather this moved into the code in v2.0.0 - that way all workflow info is self contained and no supplementary files are needed. The multiple data store thing is entirely an issue, in v.2.0.0 we are aiming to make more aspects (such as you have described) pluggable, using whatever data stores you specify. That will require a lot of docs but we are up to task.
That you so much for the advice comments, I will share them with the development team, feedback is always appreciated :)
However, all cron formats are subject to lots of ... edge cases depending on what you think of as normal. For example, since Vixie cron (and its descendants) are pattern matchers, there isn't a way to express "Run this every 33 minutes". Even a "sane schedule" gets to have fun with timezones and their adjustments (both for daylight savings but also countries adjusting their rules).
Airbnb's Chronos chose instead to use ISO 8601 Repeating Intervals [2] which in written form are R[n]/<timestamp>/<period> (where n is the number of repetitions and timestamp is the start time). That's a good format for an infinitely repeating thing with arbitrary period, but not able to actually express "at 6PM every weekday" (which crons do, and many humans want).
Finally, all repeating schedules go out the window once you get to someone who probably needs something more explicit: every NASDAQ trading day after market close (which is usually 4:30 but sometimes not, not open every weekday, etc.). The usual answer is "Just make the schedule to run every weekday at 1 PM Eastern for the early closes and usually exit, every weekday at the usual 4PM Eastern and have your code check that the market isn't on holiday". The same kind of works for the 33-minute period, except the answer ends up being "check every minute and do your own modulus to figure out if you should run" (which isn't particularly helpful).
Enjoy scheduling!
[1] https://cloud.google.com/appengine/docs/standard/python/conf...
[2] https://en.wikipedia.org/wiki/ISO_8601#Repeating_intervals
The actual full name doesn't matter that much, and it can still be called "Odin" as shorthand in the docs, but this way the full name can still have both the implications you want and uniqueness. Plus, adding "Flow" now makes it a tiny bit more self-describing than just a proper name.
If I wanted to take a stab at a Scala SDK, is that something you would be interested in supporting, or should community/contrib SDKs be managed as separate projects/plugins?
They're both also distributed workflow scheduling and state management
Specifically, the observability hooks in Odin would be the differentiating feature. Information is gathered to help infer the internal state of jobs. This means Odin can directly help diagnose where the problems are and get to the root cause of any interruptions. Debugging, but you're one step ahead.
I get you can technically do orchestration and workflow with this, but you can also technically do it with any turing complete language.
Orchestration and workflow should involve UI design elements for workflows, integration/interfaces, and visual execution tracking and workflow instance initiating?
And yeah, MongoDB, which just got torn apart by aphyr for anything concurrent or distributed.
Relatedly, how many jobs/s or queries/s do you expect Odin to support in the suggested 3-leader configuration on typical VMs? What kind of load do you run this in production?
I can't imagine how anyone can model even moderately complex use cases on a key/value store. Even with some sort of ORM filtering and manipulation would be very cumbersome.
We are currently working on having the database component as something more pluggable, so down the line you will be able to use the DB of your preference :D
An earlier discussion on why Odin: https://news.ycombinator.com/item?id=23460066