Simpler long polling with Django and gevent

prg10001.blogspot.com

30 points·denik··4 comments

I haven't seen gevent before but it looks really cool. Its also impressive to see Django running on top of libevent - something I'd never thought possible.

My big concern with this framework though would whether it could cope with accessing a database or similar datastore without blocking. My guess is that it currently doesn't.

That issue (accessing databases without blocking) is why I've recently started getting interested in DBSlayer from the NYTimes:

http://dbslayer.org/projects/dbslayer

It's a proxy that lets you make SQL queries against your database over HTTP. Since all of the event based web frameworks (Twisted, gevent, Tornado etc) allow you to make non-blocking HTTP requests, using DBSlayer could instantly resolve the blocking database query problem.

To my knowledge no one has written a Django ORM backend for DBSlayer yet, but it would be a fun hacking project.

Thanks!

Your concern is valid though, if the database module is a wrapper around C library which uses blocking sockets, each database call will block the whole interpreter, not just a particular greenlet.

This is not specific to gevent thought. Tornado and Twisted face the same problem and Twisted has a threadpool to cope with that. I guess I would have to implement something similar for gevent.

However, if the database module was a pure Python or used libevent underneath then it would integrate with gevent seamlessly. Alas, it's rarely the case for database modules (?)

MySQL Connector/Python is a pure-Python implementation of the MySQL client protocol - I presume it uses sockets, in which case the gevent monkey-patching might get it to work transparently.

https://launchpad.net/myconnpy

Reply on news.ycombinator.com