Hacker News Reader

Comment by dimgl | Hacker News Reader

Why is this so hard to believe? This is proof that simple software can scale very well. Your takeaway, IMO, should be that a lot of solutions are over-engineered relative to the businesses they serve. Not every company handles Slack's scale.

View on HN

It’s no longer simple when you have platform code to prevent nodes from disappearing or dying on the minute on a kubernetes cluster the size of slack. A triggered pulse event stream would have done the trick to invoke a lambda or call code for every “thing” that needed a beat. Kubernetes comes with a scheduler…

This is my take as well, just because you can get away with something doesn’t necessarily make it desirable to maintain or extend, and I honestly cannot imagine the effort in terms of labor hours that you’d have to go through to develop something like this compared to just plugging in an off-the-shelf scheduler to something slightly more sophisticated like k8s or even just a worker-and-queue system. When you’re talking about platform engineering to solve a problem that a relatively extensible Celery service could do (and have tests and such), I have no idea how the former could be “less work” or cheaper in the long haul.

Hard to believe because I have seen the cron dumpster fire at so many companies. Random cronjobs running on random boxes ends up with mystery jobs whose existence often goes unnoticed as people leave the org and docs are out of date. Random accounts with different cron jobs, with no visibility from one account to another. Box failure leads to mystery jobs failing with no way to figure out what happened. No “advanced” features like retry and retry with exponential back off. No database with job history. No parallelism.

Every place I have worked cron turned into a dumpster fire.

Sure, but those are organisational failures, not technical failures. If you need better technology because the org sucks at planning, the tech isn’t the issue and you are just sweeping issues under the carpet.

Or you could have a canonical cron box.

Or a fleet? Some of the problems up-thread feel organizational rather than tooling.

No matter what they tell you, it’s a people problem.

i will see your rando cron box and raise you Excel VBA macros run on Windows Task Scheduler on a spare machine in a cubicle.

Never forget that the world economy is probably being held together by a handful of these.

A boatload. Couple. Panamax or bigger. Otherwise agreed.

Cron scaling? Seriously?

By definition cron does not scale, it is account by account per VM/machine with no rhyme or reason.

Alternately, if you define cron as a starting/startup point, an expectation when scaling, then simple format for a job queue, the context of the post is understood.

Yeah that’s the point. If managed properly, turns out it can handle a $20B tech company.

Reply on news.ycombinator.com