Schedule: Python job scheduling for humans. | Hacker News Reader

Schedule: Python job scheduling for humans.

github.com

13 pointsdbader8 comments

Isn't it quite unfriendly to make the caller implement the main loop, and require periodic polling of the scheduler in order to run jobs? When I have implemented things like this before, I kept tuples of (time_of_next_run, job) in a heap. Then the scheduler had a loop like:

  while True:
      now = time.time()
      while job_heap[0][0] < now:
          time, job = heapq.heappop(job_heap)
          job.run()
          heapq.heappush(job_heap, (job.next_time(), job))
      time.sleep(job_heap[0][0] - time.time())

This reduces the amount of polling needed, since you sleep until the next job is ready.

You're certainly right that the module should provide a main loop of some sort for convenience.

I wasn't totally sure how to implement this to make everyone happy -- some people may want to use time.sleep(), others want threading.Event().wait(). Maybe I'll simply go with something like this:

  def run_forever(sleep_func=time.sleep):
    ...

While the API looks really nice and clean, this seems to lack any sort of task distribution mechanism, i.e. tasks run in-process but cannot be passed off to off-process workers, like in e.g. Celery [1] or RQ [2] (the former also does scheduling, the latter is more of a job queueing library, that can do scheduling via an add-on [3])

1: http://www.celeryproject.org/ 2: http://python-rq.org/ 3: https://github.com/ui/rq-scheduler

I use something like this for submitting jobs to beanstalkc:

  schedule.every().hour.do(beanstalkc.put, "my_job_id")

I poked through this briefly. Anyone know what advantages this has over built in system utilities like cron/etc aside from being written in python?

Using system utilities like cron, can be suboptimal if all you're trying to do is schedule periodic tasks within an application, e.g. database cleanup tasks, periodic polling of other services etc.

The main gripes with cron are:

- cron can be overkill. It requires maintaining a separate crontab file for your app. Wit schedule you can keep everything in pure Python.

- cron is per machine. Multiple app servers require locks of some sort to prevent job duplication. To solve this, we could put triggered jobs on a shared jobqueue (beanstalkc, resque, ...) and have them processed by several workers.

- cron's syntax, while powerful, can be difficult to understand and maintain. Schedule provides an easy to read syntax based on the builder pattern, e.g. "schedule.every(2).hours.do(job, args)"

Much of this was said in better words by Addam Wiggins: http://adam.heroku.com/past/2010/4/13/rethinking_cron/

I wrote schedule because I needed a lightweight scheduling solution for a Python web service backend. It's 'clockwork' (https://github.com/tomykaira/clockwork) for Python.

I was first thinking, why not just use APScheduler. But then I saw the syntax... neat!!

Thanks :-)

Reply on news.ycombinator.com