Hacker News Reader

Comment by bsaul | Hacker News Reader

Just checked the book index, and i've got one recommandation for the v2 of the book : add a chapter about asynchronous tasks / jobs using celery. This would probably fit in one chapter, and it makes a world of a difference as to what you'll be able to build.

View on HN

Can you elaborate on why celery makes a world of difference?

Celery is a system for job queues / asynchronous tasks / whatever you want to name it. Usually in web the in-between time of request and response is the time it takes for your application to do the work. However, if you plan to do a lot of work (e.g. render business reports), or the work can take quite long (e.g. make a request to an external service), asynchronous processes make a lot of sense. The request will return the response fast and the application registers a "task" in the queue. A queue system will then pop one of the tasks independent of the web request/response cycle and process the tasks in it's own time.

In short, it simply makes your application respond faster (and scale better) while you still can keep track of all the work done.

I am quite familiar with gearman and beanstalkd, but Celery (as far as I understand it) is a python layer over these implementations. It allows you to write code agnostic of the broker, which helps for portability etc.

Well, if you ever want to build any real world system other than just a blog or any corporate website (which frankly would better be done with wordpress), then you're confronted with the fact that everything can't simply be done instantly and / or upon an http request.

Think of things like monitoring, reporting, emailing, analytics, cleaning, archiving... the list goes on.

For all those things you need a job system. Celery is (to my knowledge) the most popular one.

Reply on news.ycombinator.com