Kala: A Modern Job Scheduler (written in Go) | Hacker News Reader

Kala: A Modern Job Scheduler (written in Go)

github.com

64 pointsjaytaylor31 comments

Exactly what I was looking for since months now!

Loved the architecture of Chronos, but with JVM, Zookeper and Mesos as requirements, it had "heavyweight" written all over the place. This is what the successor of cron should be IMHO. Distributed by design, API driven and dynamic.

EDIT: Apparently it's NOT distributed yet. Had hoped for Raft or etcd integration.

Feel free to throw issues into the GH tracker so we can get them on the roadmap.

My initial use case for this is for complicated time-based triggers that would be an absolute PITA/impossible to model in cron. We both felt the pain of setting up celery, etc.; @ajvb took that pain and put this together.

https://github.com/erikh/jobd uses etcd for distribution of jobs.

The initial use case was for a single node setup, Chronos for an early stage startup. I/we will be adding support for it to be distributed here soon, but not until the single-node, single binary with no external requirements setup is production ready.

If you have any feature requests, please create an issue in github. The more people request something the sooner it will get implemented.

Thanks for the feedback though.

Side note: "Kala" means time, in Sanskrit. It also means someone who controls "death". Feminine form of it is "kali". I think it is only fitting to name a scheduler after time (or something that is controlled by it). ;)

[1] https://en.wikipedia.org/wiki/K%C4%81la_(time)

Sadly in Finnish it means "fish".

Luckily in Greek it means 'good'.

Whereas in Bengali it means "black", and in Swahili it's a kind of wild cat.

Kala also means performing art in Sanskrit (and Hindi and Bengali afaik)

https://en.wikipedia.org/wiki/Kalā

depends on how you say it, if pronounced kla then it's art if pronounced kala then it means black. (Hindi)

If art it's pronounced more like kuh-LAA

If black it's kalaa

If time it's kaa-La

And in Swedish, it's the plural positive of kal, which means bald.

Well, it means "well". Like, "well done". Or, "good <things>", plural.

Interestingly, in Turkish it means "until".

It was taken from the Sanskrit meaning for "time". :)

Looks good. Very much looking forward to this being production ready. I am currently looking for a scheduler that provides a better UI/UX than cron alone.

Currently, what do you all use for this sort of thing?

Nextdoor has a not-yet-open-sourced solution: https://engblog.nextdoor.com/2015/06/10/we-do-not-run-cron-j... that looks great.

Does anyone have any experience with Yelp's tron? https://github.com/Yelp/Tron

I work with @ajvb. We'll be working on getting this into production over the next 2 months. To do so, we'll be working on adding redis as a backend and making it distributable. The goal is to make it really easy to get started, but be able to scale out to bigger production systems easily.

@ajvb has plans for an awesome dashboard as well to complement this :) Stay tuned... good things ahead.

Is this associated with http://www.222labs.com ?

If not, is there an email or somehow I can PM you or @ajvb at?

We work together there. We actually both started working on a job scheduler independently, but joined forces and he ran with it. It's @ajvb's personal project, but we'll be using it at 222 labs for sure. You can shoot me an email at my username without the numbers @ gmail.com and I'll hook in @ajvb to the thread.

Please add postgres support.

Shouldn't be too hard. I threw in an issue to track it: https://github.com/ajvb/kala/issues/41

With pleasure.

I had not seen Nextdoor's scheduler, which looks awesome except for the required external requirement of SQS. Kala was built to not require anything except for the Kala binary.

Yelp's Tron is awesome, but I wanted an JSON over HTTP API and ISO 8601 Duration Notation. Plus there are a few over problems with it that I currently don't remember.

This looks good, nice work.

I need something like this at the moment, but I'm curious what the target company size is for a project like this. For example, we're a 2-3 person team and not quite big enough to have the capacity to run our own deployment of something as critical as Kala/job schedulers. Nor do I want to run another DB like Bolt/Redis when I'm already running another critical DB.

I'll probably go with a hosted solution ala http://www.iron.io/worker until we can run our own.

Bolt is an in-process database, like sqlite or leveldb. And it's native-Go, so there are no dependencies. You don't need to run additional service infrastructure to use it.

Just for clarification, the IronWorker platform by Iron.io supports large enterprise deployments in cloud, on-premises, and hybrid environments, so it scales from small hosted projects to very large deployments processing millions of jobs per hour. http://www.iron.io/enterprise/

dmix, a 2-3 person team is actually exactly what this project was built for. The use case came from a start-up of the past that I was an engineer at that really really needed something like this.

Fair enough, different teams have more resources for sys admin work or are at a later stage where they can invest in it. I'm just wary of adding any external pieces of software after being burned a few times staying up late on weekends fixing a broken service.

is there way to migrate existing jobs from tools like jenkins to kala ?

I'm sure it's possible, but it's marked as 'Alpha' in bold letters, "not production ready".

Can you start an issue so we can start discussing what this would look like? What kind of APIs would you need for this?

Reply on news.ycombinator.com