Hacker News Reader

Comment by twic | Hacker News Reader

Really interesting! It would be great if the readme could go into more detail about how the relationship between the hub, CLI, and cron works, including an explicit description of what text goes in the crontab files.

How do you get the reports of each run back to the hub? Are you wrapping each cronned command in a script which captures the output?

Does the hub have its own notion of what jobs exist on each machine? If so, there is the potential for this to get out of sync with what's really on the machine (manual editing of crontab, database crash and restore from an old backup, etc). Is there any way to detect that, and bring them back into sync?

Can the hub form an opinion on whether any job has failed to run when it should? Can that be exposed to something like Nagios?

How do i integrate this with rcron for my highly available cronjobs?

Have you thought about supporing sqlite for a database? It would simplify deployment considerably in simple cases.

View on HN

Sure, that's a pretty important bit to understand so I'll try and add some more information on that.

Yeah, so to convert a cron that ran `ls` to use minicron you would change it to `minicron run ls`. Currently the full command that gets added to the crontab is something like `/bin/bash -l -c 'minicron run ls'`, I use /bin/bash -l so it's easier to get working when you are using something like rvm for managing ruby versions. The command then gets run by minicron in a pseudo terminal so I can capture output line by line or even character by character and send it on.

At the moment the hub only knows about jobs which you create via it or for when you first run a job it gets created, so if you ran `minicron run ls` manually from server1 a job called ls on server1 would be stored in the db. Currently there is a potential for them to get out of sync yes, I had some ideas about how a push/pull type sync feature could work but I ran out of time to work on it. It's definitely something I want to improve in the future though!

Yeah, so the hub knows about any 'schedules' i.e cron expressions that you set up from it, so if you set up a schedule of '* * * * *' i.e every minute and that cron doesn't execute every minute it will send an alert via email/sms/pagerduty if you have enabled them. It does this by polling the database for executions in the background. I'm sure a way to expose those alerts to Nagios could be added.

I hadn't heard of rcron, I will have to look into that.

Yes and I agree! I did have SQlite support at one point but I dropped it because I was using triggers for cascading deletes, I'm not doing that anymore though so it should be possible to add it back pretty easily.

Thanks for your thoughts!

Thanks for your responses. This all sounds really sensible.

Concerning rcron, you don't actually need explicit support for it, as it's just part of the scheduled command. However, some integration could make it obscenely pleasant to use. Probably, i would need to manually set up rcron on a group of servers, then configure some sort of 'virtual server' in the hub which contains the servers in the group. When i schedule a job on the virtual server, it is added to the crontabs on all the real servers, but guarded with rcron, so it only executes on one. The reporting would then deal with the fact that only the master in the group actually runs the job: if the rcron wraps the minicron, then it would deal with the fact that only one server sent a report; if the minicron wraps the rcron, it would just hide the empty reports from the non-master servers.

We currently manage most of our cron jobs through Puppet. If we adopted minicron, we'd want to keep doing that. That means we would write a Puppet defined type which adds a job in the hub. It would be really useful if there was an API, or ideally a command-line tool, to do that. It would be nice if there was then an easy way to distinguish Puppet-managed and interactively defined jobs in the hub, so that people didn't attempt to fiddle with the Puppetted ones. Maybe a flag that makes them read-only in the UI, but allows them to be managed in the API? That's probably something we would address in a fork if we did adopt minicron.

Some kind of auditing for changes to job definitions would be useful. We currently use Jenkins for ad-hoc scheduled jobs, and our very weak audit trail for it is a recurrent pain point. Would it make any sense to keep job definitions in a Git repository? Might be quite a pain to provide an interactive UI on top of that, though.

Thanks for the info on rcron, it sounds like a really useful tool. I think I prefer the approach of rcron wrapping minicron so it doesn't require added complexity to minicron. I've added an issue here for potentially doing this https://github.com/jamesrwhite/minicron/issues/66 if you want to follow along with the progress.

The fronted is actually all API based, in theory I don't see any reason why external apps couldn't use the API. I'll try and add some documentation for it. I quite like the idea of puppet/chef/whatever defined jobs. When I add permission support to the hub/api you could set up a user for puppet which would have its own API credentials so the API would be able to show who the job was created by. On of the features of the permissions system will most likely be that users can only edit jobs they created unless they are an admin or some kind of user with extra privileges. I think this feature would be useful for a lot of people so hopefully it can be added without forking the project!

Logging activity and some kind of revision control is definitely something I've thought about and plan to add in the future. I remember seeing a ruby gem a little while back that could basically track changes to arbitrary ActiveRecord models, that might be useful here. I've added two tickets for this to remind me! https://github.com/jamesrwhite/minicron/issues/67 + https://github.com/jamesrwhite/minicron/issues/68

Reply on news.ycombinator.com