Rethinking Cron | Hacker News Reader

Rethinking Cron

adam.heroku.com

77 pointsjfi27 comments

Meta point: I'd really like to see (non-www) subdomains added to the article title. I clicked through to the article thinking that Heroku had redesigned their Cron management only to discover something completely different. Subdomains are included in Posterous links; why not everywhere?

I generally agree with the weak points that the OP pointed out regarding cron. That being said, throwing out cron, or any other built-in system tool, isn't always a great idea.

When you adopt a new scheduler, you're inevitably creating work for your enterprise. It's yet another thing that must be learned and documented. There may be a new syntax to learn, which will mean you're incurring training costs (and the cost of errors) down the line.

Programmers love to improve things, but there's a real world cost to change that can't be ignored. cron may be archaic and lacking power, but it's well understood.

As for enterprise schedulers, they've been around for years. Off the top of my head, I've worked with 'Autosys', and 'Appworx'. These were robust, enterprise-ready schedulers that supported conditional execution, locks, as well as their own scripting language.

To be fair he didn't say throw out cron. He said cron will have a place for rotating log files etc. Which to me is infrastructure type stuff and not application level.

So the sysadmins will still use cron to do the infrastructure type jobs. But when a programmer needs something in his app to be scheduled he can use the replacement, this also frees them up from asking the sysadmin to install a cron script.

And the subtle differences between a cronjob’s shell environment and your command prompt’s shell environment can be maddening.

This is my favorite thing to complain about with cron. I can't believe that there's still not a way to test run cron entries. So far I just add > tmp.out >> tmp.err, set it to run every minute, and wait.

You should set some variables (SHELL, PATH, HOME) in your crontab. Just run "sudo env -i SHELL=... PATH=... HOME=... cmd"; this should work if cron would work.

http://serverfault.com/questions/85893/running-a-cron-job-ma...

I completely agree that there should be a better, direct way to test cron jobs.

I used to do this by exec()ing a shell with no environment. 'Twasn't perfect, but it caught a lot of irritating little errors.

At Bizo (http://dev.bizo.com), we manage scheduled tasks via Hudson.

It works great for us. It's centralized, has a web interface, and is already familiar to everyone on the team (we already used Hudson for CI). It has a ton of plugins, so notifications, single sign-on, and a host of other desired features were no problem to bolt on.

Here's a quick post from a few months back on why we chose Hudson: http://dev.bizo.com/2009/11/using-hudson-to-manage-crons.htm...

How about http://upstart.ubuntu.com/

Upstart is more for replacing init. Its less suitable for jobs that need to be started at particular times since its designed for starting and stopping services during boot and changes in runlevel.

I think long term plan for upstart is to replace cron[1](generating events at scheduled time ...)

And some more general information here [2]

[1] http://upstart.ubuntu.com/faq.html#replace-cron

[2] http://www.netsplit.com/2006/09/01/upstart-can-now-replace-s...

Ooh - hadn't seen that - looks interesting. I stand corrected.

Hopefully they do a better job than Apple. I had a number of issues with their init/cron replacement.

so true! Every time i'm forced to deal with launchd for scheduling, I wish I was using cron.

This is what I am looking forward to. Cron, atd, and init all have functionality that should be integrated and udev aware.

And this functionality should be easily available to all applications. If I want to send a specific email at noon, or backup my laptop drive every night at 3am only when on my home network, etc., I should be able to without using 5 different daemons or programs kept running in the background and hacked-together scripts.

It would be easier to use following cron syntax to run things at reboot :)

  @reboot /run/a/program

I'll reprise my comment from the last time a new and better cron was posted here[1]:

While cron is clearly pushed way past its capabilities by many people (without always realizing it), this strikes me as a reinvention of something that's been around forever: the batch system.

If the goals are improved reliability, management, fault-tolerance, etc., you'd be better off using mature software that was designed to solve this problem, and already has good community support, like Condor.

[1] http://news.ycombinator.com/item?id=916737

I developed Whenever (http://github.com/javan/whenever) to at least make the Cron syntax more reasonable.

Whenever is awesome.

(incidentally, check your pull requests :)

OP wants a distributed operating system. Linux, FreeBSD and Windows by design are not distributed. POSIX specifically cripples this. How do you take into account an action that needs to be distributed but have a central return point? How will fault tolerance be handled? Expecting every application to handle this won't work, what's needed is a distributed operating system.

launchd

I'm genuinely confused as to why launchd isn't part of this conversation. It is definitely a replacement for cron (and init) and seems to be a pretty good one. I haven't done an extensive comparison between it and cron, but it has definitely been easy to use and worked well for me. It is very flexible and you use it by laying out exactly what you want it to do in XML.

I had issues in the past with scheduling processes in launchd. It's been a while, but I remember having these issues at least two OS revisions after Apple initially launched it. (The issue was related to starting a process and having launchd restart the process if it died. There were options to toggle this but it wasn't working for me, and I was reading the Apple developer docs on launchd.)

Also, I personally don't feel that using XML as a config file format is necessarily an improvement over what cron offers now. (I know that I'm skipping the fact that 'XML is self-validating,' but I don't really find a lack of a self-validating config file to be a major issue with cron)

One of the biggest gotchas I found was that launchd bails if the process takes less than 60 seconds to run. Putting a "sleep 60" at the tail of the script fixed a lot of problems for me.

At first I was kind of upset that this condition was there, but in retrospect it might have been a very wise design choice to stop runaway triggers.

Now that you mention it, I remember that quirk. I had an issue with not being able to disable that quirk when running a scheduled process. launchd would stop scheduling a process that was scheduled to run every 5 minutes if one of the runs took less than 60 seconds. It really irked me because this 60-second-rule only makes sense when you are talking about re-spawning a constantly running process, and should not affect the behaviour of processes running on a schedule.

> At first I was kind of upset that this condition was there, but in retrospect it might have been a very wise design choice to stop runaway triggers.

It doesn't really stop runaway triggers if all of the scripts have a 'sleep 60' at the end of them. And if you are concerned about runaway triggers you could always make the rule something like "bail if the process dies after <60s more than once in a 2-minute period" or similar since there are legitimate processes out there that take less than 60 seconds to run.

Well, I didn't say it was perfect. =)

No mention in the article about Gearman for worker queues so I'll promote it here. I like its simplicity and language agnosticism. http://gearman.org/

Reply on news.ycombinator.com