Tornado: FriendFeed's non-blocking Python web server is now open source
bret.appspot.com
bret.appspot.com
Thanks a lot, fellas. Asynchronous networking programming in Python is somewhat of a bear so it's really nice to see a tool get released that improves it!
Website: http://www.tornadoweb.org/
CherryPy did max out the CPU on all of the cores in the load test, so I think it was a fair test.
I'm intimately familiar with the limitations of python threads - and right now I'm the maintainer of the multiprocessing module, which is the process-based "reply" to the stdlib threading module.
Even cpu bound tasks can release the GIL to do processing(for non python api tasks).
It's clear that Facebook wasn't using a single Apache/mod_wsgi process because that wouldn't get close to 2,000 requests per second. I'm sure they gave it the number of processes that were appropriate for the amount of RAM on the box - the issue is that every Apache process (of which I'm sure they dozens if not hundreds) uses up memory.
It's hard to set that up properly and that's what makes benchmarking so hard. Still, it's not hard to believe that with the weight of Apache in the process, there's going to be at least 5MB of overhead for every request. If you were saving that 5MB of RAM per client times the 2,223 requests per second, that's 11GB worth of freed RAM per second. Let's say Django uses 10MB of RAM for a request: that's an extra 50% increase in capacity right there just by eliminating the 5MB Apache process overhead. And Apache tends to be on the heavier side (5MB is somewhat low as an estimate) and if you can do it asynchronously, you're not tying up RAM as you wait for clients, so you do well.
Now, put Apache/mod_wsgi behind nginx (as a proxy) and you'll also see an increase in requests per second because you won't be tying up all that RAM as a client is downloading and only use the RAM while actually doing work in Python freeing that memory to be used elsewhere. However, I'm guessing that they ran this benchmark locally and so the network tying up Apache instances wasn't a factor.
Clearly, benchmarks are benchmarks, but it isn't as if Facebook and FriendFeed don't have lots of smart engineers and they wouldn't be using their own server if Apache was significantly better. They have a site of massive scale and know how to benchmark things since you need objective data to make decisions based on rather than the religious arguments some get into.
I'd guess that, if one could get Django running under Tornado, it would run similarly and that it's more of Apache being the bottleneck/RAM hog.
IApache's only saving grace is that you can handle everything monolithically (albeit with mediocrity).
Just played around with it a bit and it's basically what I want:
1) Simple and clear syntax (e.g. they use 'end' not endfor, endblock, etc)
2) Assign template variables to anything (including functions)
3) Don't over-restrict the author (e.g. they allow list comprehensions in if tags)
4) Block and extends statements.
Error handling seems to be a little clearer than other template languages though still not great:
Clearly it's not going to be for everyone but after slogging around with Mako for the last year (<%namespace:def /> tags anyone?) this is like a breath of fresh air.
Now all we need are async data clients that tie into the Tornado event loop, and a clean way to yield control back and forth (perhaps with coroutines - http://www.python.org/dev/peps/pep-0342/). Unfortunately, I don't think there are any async Python mysql clients (PHP has one now - http://www.scribd.com/doc/7588165/mysqlnd-Asynchronous-Queri...)
A web app written like that would run like it's on fire. The event loop would just roll through everything asynchronously. You could even automatically batch together data calls across http requests to make things easier on the data tier.
For other web apps though, the frontend-backend delay is comparatively more important, because usually there's a load balancer or reverse proxy in charge of buffering up data arriving from distant/laggy browser connections and hitting the web application server with it all at once.
[0] http://mapleoin.fedorapeople.org/pkgs/tornado/tornado-0.1-1....
A few notes: - template.py: "Error-reporting is currently... uh, interesting." - url mapping looks primitive - no form helpers - database.py looks decent, but is mysql only - Very nice to see the security considerations in signed cookies, auth, csrf protection.
Overall, it looks like they've done the trickier parts of building a web framework and left it to the user/community to add the parts web developers use most frequently (form & url helpers, code organization, and orm).
I love Facebook's enlightened approach to open source. If only one of their open source projects takes off, it will benefit them tremendously.
Obviously missing features (auth, nicks, scrolling) but that can all be added in a few mins.
It seems that the main advantage of this is that you have one thread manging many sockets. I am a bit surprised that blocking kthreads would be so much slower relatively. What causes the slowness? Context switching? Additional stack memory usage?
With an event driven architecture you only have to hold the session information per connection in memory which can be as low as 4K, therefore you are able to maintain (depending on the complexity of the protocoll) n*100K connections.
The only drawback that you have to write and think your whole program event driven; you write callbacks for each io and timer operation and the control flow won't be clear if you read the program.
Anyways, I was talking about C, where the stack is allocated when the thread is created and you cannot resize it later, nor it will grow automatically.
With dynamic languages it is different, but don't expect handling that much connections with them either.
Typically an output buffer should be able to hold the complete production for a client, if you limit it to say 4K you will be unable to process a request in one go.
I realize Tornado isn't any different from the other Python frameworks with regards to coding style, etc. but the the fact that this framework comes complete with a web server means I don't have to worry about that part of the equation making developing Python-based webapps almost identical to developing a C++ library with one of the many HTML-based UI frontends :)
Having a great time playing around with it... almost done with a basic forum system built on Tornado + Storm (https://storm.canonical.com/).. I think I'm getting the hang of this whole web-development thing! :D
When we were developing this, we found that Twisted introduced as many problems as it solved in terms of incomplete features and bugs. The other protocols seem to have more attention than HTTP from what I could tell.
I'm not sure even that is true. When I used twisted to write the justin.tv chat server (which is essentially an irc server) I gave up on twisted's irc protocol implementation and wound up just using twisted as an I/O layer.
This is sort of the point. twisted isn't a web framework. It's a networking framework. I work on a lot of really awesome networking projects that either don't have web interfaces (various xmpp services) or have ones that need work (buildbot).
Instead of filling an obviously missing hole in an existing framework, a new one was created that is missing all of the stuff that the other project has.
The twisted http client stuff is quite good. I used it along with a friend to build a tool that is a realtime (in the web sense) gateway between friendfeed and xmpp clients a couple days after the realtime API was released (before friendfeed launched their own). It works very well and I still use it today.
Unfortunately, if I wanted to build a tool that did similar stuff with a Tornado front-end, I'd have to write a new HTTP client that's compatible with Tornado's event loop (unless I'm mistaken).
Everything comes at a cost. Tornado demonstrably solved Friendfeed's problems quite well, but, as I stated above, I can't use it to solve my problems because even if I rewrote my apps, I'd lose all of the rest of twisted upon which I'm relying. e.g. I can't just bolt it onto buildbot (which would be so ideal as our web interface is suffering from lack of a framwork, but the backend is really good).
It's not OK for things like twitter's realtime APIs that provide infinite streams of data.
However, even in the case of friendfeed's API, I process the data incrementally. It's not uncommon for users of my service to receive data via xmpp before the http request has even completed.
I also use this technique for twitter's non-realtime APIs -- I use a pull-based SAX parser (that comes with twisted) to incrementally process the stream and collect and pre-sort the interesting parts. Then one of the callbacks I attach to the completion of the request (note one of: I have several that are reusable components) delivers the pre-sorted, pre-filtered results out via xmpp. I did this to reduce memory used in my daemon. It really helped.
So while it has one, it doesn't appear to be full-featured enough to work in my applications. And this is the sort of cause of confusion here. Twisted has a great network stack and a ton of really awesome protocol implementations sitting on the shelf (my apps mix http client and servers, xmpp, couchdb, dns, finger, etc...). What it doesn't have is a decent web framework.
One of the twisted guys, however, said it should be possible to transplant the web framework part of tornado onto twisted, so that would be great for the rest of us.
Does this web server offer any specific gains? Or is it just a question of personal taste?
Suffices to say, no, we do not support that :)
It seems like this would not be mandatory in Tornado, since it is an app server; from my reading I'm assuming all pages are dynamic.
ESI: http://www.akamai.com/html/support/esi.html
e.g. When you and I log into ff, simplifying greatly, that page would say, "Welcome, Bret" or "Welcome, Prakash". If you could abstract away "Bret" and "Prakash", that page would be cacheable. And, the db hit is only for that particular fragment.
You would need to re-code the pages with esi tags to do it. The other way would be to use javascript to do the same.
What do you think?