Multiple Django and Flask Sites with Nginx and uWSGI Emperor | Hacker News Reader

Multiple Django and Flask Sites with Nginx and uWSGI Emperor

tghw.com

142 pointstghw26 comments

This is my favorite stack too. I usually use supervisord to handle uwsgi processes but this looks much more debian/ubuntu-like. Thanks for the write up. I like to think I am pretty experienced with Django/Python but it's ALWAYS fun to learn new things. Especially when sometimes it feels like a lot of the innovation and new stuff is occurring elsewhere (cough node)

Don't worry much about node, Twisted still beats it for async workloads and PyPI is still much bigger than npm.

I'm a big fan of twisted too. Can you show me the benchmark for Twisted vs. Node for async workloads?

I went looking, and found an experiment someone did a couple years ago, where he wrote a trivial "echo data over a socket" server and found that pretty much everything held up okay.

http://oddments.org/scalestack-vs-node-vs-twisted-vs-eventle...

Obviously this doesn't tell us much; a better comparison would be nice.

They'll both be bottlenecked on the system polling mechanism, barring any obvious deficiencies.

They're both very fast and microbenchmarks won't show anything very relevant.

What a coincidence... I just blogged about our switch to uWSGI. For anybody interested check out our setup here: http://amix.dk/blog/post/19689

As you can read in the above blog post there are some issues, especially when handling very big GET requests (you need to set a bigger internal buffer) and when handling uploads (you need to extend the default send and receive timeouts). Other than this, uWSGI rocks and I highly recommend it.

I enjoy seeing these python related posts being submitted. As @whalesalad said, it seems a lot of the innovation is in ruby/node land.

Although the first thing I noticed when I clicked the link was that it looks like it's a scraped design of Svbtle.

edit: spelling.

Yea, it is a scraped design. https://github.com/orf/simple

I was more surprised that the author claims this as a new feature. I feel like I heard about this over a year ago... In any case, excellent write-up.

Could someone please explain to me what uwsgi does? Looking at the main page, it does ... everything! It's a wsgi server, but it looks like the default configuration is still to put it behind another server like nginx or cherokee. Also, nginx for example, has its own mod_wsgi. After reading the docs, I still don't quite understand why I'd want that extra layer between my server and my code.

Take it as an "infrastructure"/"swiss army knife" for hosting and deployment. Most of its features are still topic of research in autoscaling,management, tuning and so on... Most of the users do not need that features, but others (like the Emperor) can be useful for everyone. Regarding the additional layer, is something required, as your code/framework need a way to communicate with the webserver and being healthy. Nginx's mod_wsgi is an old non official module, very funny, but it works in a very different way embedding python itself in the nginx core.

I see, thanks!

Are there any updated benchmarks/feature comparisons between nginx/uWSGI vs. nginx/gunicorn out there?

The web server doesn't matter. Your app is what will be slow.

Personally, I would like to know if this setup is available using gunicorn.

It would be very similar, you'd just need supervisord or something similar to run gunicorn for you. There is a pretty complete example of deploying gunicorn + nginx in the docs: http://gunicorn.org/deploy.html

So you can serve a production site using Python's wsgiref and SimpleHTTPServer?

Servers do matter - memory usage, throughput, error rate, concurrent request handling etc. If the options are to use a faster server, or hunt down your code for bottlenecks, the choice is clear. Even if I modify my code to be faster, a faster server doesn't hurt.

This "benchmarks are useless" meme is overdone. "Hello World" benchmarks don't matter, but if a server is significantly faster on my real world application, why would that benchmark be meaningless for me?

I'm actually setting up a new ec2 instance tonight as my current one is set to expire.

I currently run DUNG (nginx/gunicorn), perhaps I will run nginx/uWSGI first and compare performance. I will ping you if I get around to it.

Your wsgi server is almost never the bottleneck - see this talk by Graham Dumpleton at Pycon - http://youtu.be/Bt2HStzaBzE

There are benchmarks around [1], though I consider them valuable only for the entertainment value.

The great thing about uWSGI/Nginx over Gunicorn/Nginx is that Nginx speaks the uwsi protocol natively[2], whereas with Gunicorn Nginx acts as a proxy.

Also, in my experience (2 years with gunicorn, 1 with uwsgi in prod), uwsgi is more stable, uses less memory, has better management and configuration tools and even lets you run multiple Python apps under the same intepreter (multitenancy) [3].

[1] http://www.peterbe.com/plog/fcgi-vs-gunicorn-vs-uwsgi

[2] http://wiki.nginx.org/HttpUwsgiModule

[3] http://projects.unbit.it/uwsgi/wiki/TipsAndTricks

(Edit: formatting)

The uWSGI ppa is not really maintained at all. The recommended way to install it is via pip. It's just one binary and you're making the upstart conf yourself anyways.

Great write up and timely for us as we are looking at setting up our stack. Please keep us posted with your progress and any ongoing operations issues.

I'm somewhat confused. If both the django and flask app are running and you visit "mygreatstartup.com", which app will you see?

Since requests to mygreatstartup.com go to nginx (and get routed from there to any back-end servers you may have), that depends entirely on how you've got nginx configured.

Reply on news.ycombinator.com