Hacker News Reader

Unfortunately, the purity of simplicity here is deceptive and doesn't scale well to the real world where some jobs are resource hogs (job variance) and hang (nondeterminism and error handling). This is reinventing an HA job scheduler but without retries or load leveling. If you want even working sets without hotspots, then you need a job scheduler aware of workers' load state. This cannot happen by blindly rolling dice and giving it a cute name.

View on HN

Thx for your feedback. Most of the workload still happens inside a job queue however for this case we deem that it’s not necessary. Fortunately it’s working well for us, the task async part is also adaptable if the problem becomes more complex than what it needs to be then we can handle those cases accordingly. However we still believe throwing everything in a job queue isn’t always optimal either.

For error handling when the task fails it sends a message to the genserver we can also use that opportunity to handle retry, or do a strategy change where after first failure we throw it into a job queue.

This way we optimize for user experience and at the same time have a robust strategy for handling failure. I guess that can be another blog post on its own.

Reply on news.ycombinator.com