Python vs. Node vs. PyPy benchmarks | Hacker News Reader

Python vs. Node vs. PyPy benchmarks

blog.kgriffs.com

63 pointschanux53 comments

Don't miss this follow-up post, though:

http://blog.kgriffs.com/2012/12/18/uwsgi-vs-gunicorn-vs-node...

Graph 3 as a log y chart would've been nice.

I'm actually quite perplexed as to why the blog author chose to use the WSGIRef which is known to be one of the slowest of the bunch according to this much more thorough benchmark:

http://nichol.as/benchmark-of-python-web-servers

Even more perplexed, why wasn't the payload and the WSGI apps sources published?

Your observation is good. Author should clarify this. But in his next articles he shows benchmark against other web frameworks, which shows more justified results

That doesn't really answer why use the slowest one with PyPy (and no, uWSGI does not work for PyPy, but a lot of stuff does)

uWSGI works with PyPY (Obviously with lot of limitations on the features). http://projects.unbit.it/uwsgi/wiki/PyPy The problem is that for the vast majority of the webapps there is (still) no advantage. I only wanted to focus on that as i have invested days on (really complex for me) pypy sources, and i do not want people to ignore it ;)

His articles don't justify anything. I don't know what his payloads are or how he produced them in the apps. For all I know these 2 "benchmarks" are unreproducible. The author said he wanted to try science, there's nothing scientific about these 2 articles at all.

I don't understand why there are no numbers on these graphs. Graphs are pretty pointless if you don't label the axes.

Isn't labeling axes the first rule of graphing? This is bad.

It's either fixed or there's a problem on your end.

Nope, I don't get any numbers/labels either. Seems like the website is incompatible with a bunch of browsers.

He's just using Flot, which a lot of us use for canvas-based charts. Do they work for you at all?

http://www.flotcharts.org/

If not, what browser are you using? It would be nice to know if Flot is not compatible with something.

iceweasel 10.0.12 (basically the same as Firefox 10.0.12) It ships as the default browser for debian wheezy. The home page of flotcharts.org does work correctly.

I also tried Konqueror Version 4.8.4 (KDE's browser) and it didn't even display the graphs (not in the blog post or on flotcharts.org).

works with IE9 and firefox 18, not with chrome 24.

Great, thanks.

This whole test says more to me about the performance of non-blocking MongoDB drivers than it says about the underlying performance of the frameworks/languages. The python tests lacked a non-blocking driver. Why not use the PostGRES drivers ? That would give a much more accurate and fair treatment.

And for the record, I don't care one way or the other which is "faster," just make a damned test where one isn't given such a clear leg up.

Another set of benchmarks that shows Erlang's cowboy coming ahead on EC instances. It also compares EC2 vs real hardware and throws more contenders in the mix.

On physical hardware the race is pretty close (except node, python-gevent, but cluster node did very well).

http://eric.themoritzfamily.com/websocket-demo-results-v2.ht...

Yeah, because of cowboy and Erlang's design it is going to out perform a single process evented server hands-down.

  1. The evented servers are bound to one CPU, Erlang will use all CPUs 
  2. any bit of blocking code will stall the other code, even it just for a moment.

It is a little unfair to compare the two approaches.

I am not sure it will always under-perform with one CPU. It depends if async threads are used and if kernel poll is enabled. Erlang is designed for better low latency response at the expense of _some_ sequential slow down. I wonder if Erlang would scale slower but it will throw less errors as number of requests increases.

EDIT:

Also it would seem benchmarking Erlang and forcing it to run on a single core just like node.js is deliberately handicapping it. One of the strengths of Erlang is exactly the ability to take advantage of multiple cores.

Things like this DO NOT MATTER.

Facebook was built on PHP and MySQL, both of which are awful pieces of crap.

Twitter went down like every week for a year. It was written on RoR by people who didn't really know RoR.

Meanwhile, a bunch of better engineers using better technology got their clocks cleaned while the winners eventually had so much money they could afford to hire an army of engineers to rewrite everything.

If you know Javascript really well, use Node (although I feel sorry for you). If you know Python really well, use Python (yay). If you don't know anything, well, pick something. If you are choosing a programming language because of how many QPS you can get on some trivial task, you are an idiot.

You know the graph I would like to see? A scatterplot of how good various startup founders and early team were on engineering vs finding a market fit, with the dots labeled with how rich they are now.

Just FYI, in my own tests I've found Node.js performing about 50% better (req/s) than Tornado/C-python, while Tornado/pypy comes very close to Node.js' performance. Tornado is one of the fastest Python web frameworks and does not use WSGI (by default).

The point is that pypy can crunch almost as fast as Node.js, so while Node.js may set the standard, Python is not bad choice either from performance perspective.

I am surprised that node.js performs significantly better than python & pypy. Can anyone have some explanation?

Money.

It depends on the web framework. As the author of the benchmark shows in a later blog post [1], Python+uWSGI can be faster than Node.js

[1] http://blog.kgriffs.com/2012/12/18/uwsgi-vs-gunicorn-vs-node... (link originally posted by jdub in this discussion)

He carefully chose to use a really bad combination for pypy (or uncarefully, always assume stupidity over malice). Fast options for PyPy are cyclone.io or tornado (or eventlet or gevent on a branch I'm working on, although it's not done).

For Node.js it's a big difference between 1k and 64k files, what about something in between, like 32k files?

Red and green side-by-side on a chart... noo :(

http://jfly.iam.u-tokyo.ac.jp/color/#pallet

http://tools.medialab.sciences-po.fr/iwanthue/

IMO its a little bit excessive to ask that every graph generated is color-blind safe...

Isn't there software for colorblind people to correct that? (Something that changes the palette of colors displayed...)

Red & green is the stereotypical colorblind test, that's a pretty obvious combination to avoid. It's easy to choose colors that are at right angles in the color wheel, and that's not how accessibility works - you can't require every paraplegic to have a special wheelchair that can climb stairs, despite it existing...

Where's the source code? I'd like to see how long it takes someone who doesn't know either language to figure out what's happening in the code (a 'new' kind of response time).

If you are using wsgiref don't forget the patch

    __import__('BaseHTTPServer').BaseHTTPRequestHandler.address_string = lambda x:x.client_address[0]

I'd love to see something on the JVM against this.

Edit: pointed out in other comment

Node/V8 is a beast and will dominate web development in few years.

EDIT: Downvote me, I don't care.

That'd be true if performance would trump productivity in web development. On the other hand, I don't see a lot of web-apps written in C...

Give Node s try before judging, Node gives you same productivity as other popular languages used for web development but with C-class performance

Ummm, where are you getting that node has C-class perf?

A quick google [1] results in this article. Look for JS:V8 - Node runs off of that. Most of the graphs I glanced at show it's very close to C. I glance because I've properly checked out other benchmarks which confirm this.

[1] http://attractivechaos.github.com/plb/

If the web development industry moves from PHP to JavaScript as the main backend language... well, from quality perspective nothing will change. IMO.

If one company gave you a PHP job, and another a JS backend job, and that'd be all you got, would you flip a coin to pick one?

Or is this a way to say that you feel more satisfied with your work in Python than with your work in node?

This is my way to say several points:

- If the majority of our industry can make the switch to JavaScript, they'll ruin it in terms of average code quality.

- Even good, idiomatic JavaScript code sucks for reasons inherent in the language.

- I wish people wrote just good enough Python or Erlang or Haskell or whatever code instead of excellent PHP/JavaScript.

So yes, I treat PHP and JavaScript backend jobs equally.

[Edit: grammar.]

>> Even good, idiomatic JavaScript code sucks for reasons inherent in the language.

Care to elaborate?

I'll just throw a couple pet-peeves of mine in here:

- No proper packaging/namespacing

- Completely idiosyncratic results when operating with mixed data types (10 + 'p' = '10p', 7/0 = 'infinity' and all that)

- Variables default to a pseudo-global scope

- Prototype based 'inheritance', while cool, is just cumbersome and inelegant compared to languages that have proper object oriented language constructs

- Take all those and a syntax that was partially taken from Java, and you end up with verbose code jumble that is way more difficult to read than equivalent Python/Ruby (even C#)

- No proper packaging/namespacing

http://nodejs.org/api/modules.html

- Completely idiosyncratic results when operating with mixed data types (10 + 'p' = '10p', 7/0 = 'infinity' and all that)

I wouldn't call it idiosyncratic but just use Underscore and you are fine.

- Variables default to a pseudo-global scope

just use 'var' when declaring them

- Prototype based 'inheritance', while cool, is just cumbersome and inelegant compared to languages that have proper object oriented language constructs

Any OO is cumbersome and should be used only when it makes sense. If you don't like prototype based inheritance you can still use traditional OO with extra libs.

- Take all those and a syntax that was partially taken from Java, and you end up with verbose code jumble that is way more difficult to read than equivalent Python/Ruby (even C#)

Sorry to disagree again but there are many people preferring bracket based languages over those omitting them. You can quickly identify blocks and navigate through the source. And JS isn't verbose like Java.

JS has its quirks like any other languages but these are just very few and you get tons of amenities in return.

I don't mind languages using brackets. I used to do a lot of C and C# development and it never bothered me. The problem is that, at least to me, a Javascript code base of around 2000 lines already becomes unreadable, specially when you consider some common patterns like 'declare a function to define a namespace'. You normally end up with functions inside of functions, calling jQuery event handlers by declaring yet another inline function... By the time you use a 'for' loop, you already are 4 indentation levels/brackets in. Messy.

As you say, libraries work around a lot of these problems, but most of these workarounds are hacks. 'require.js', for example. And you shouldn't need to use underscore to get sane typing. About OO... It's debatable, I think languages like CoffeeScript and TypeScript have done a good job of abstracting the weirdness, but that's the thing: they are languages that compile to Javascript.

Anyway, I like Javascript, don't get me wrong, it's just that I don't think it's a great choice for big codebases. I know I get a lot of hate every time I say this, but to me Node.js is an excellent example of using the right tool for the wrong job. Had the energy been put behind developing the same concept as a library to a language with a decent standard library, so much work would've been saved 'catching up'.

JS is a modern and beautiful language, comparing it with PHP is bold. People fighting Node are just afraid to change and to start from zero again, they fear the truth.

No other language offers C-class speed, that rich ecosystem (which is fully async) paired with such an easy approachability plus the best and most modern package manager around. Speed is not everything but users won't tolerate unresponsive web services based on sluggish language implementations and cumbersome frameworks anymore, it's not 2005.

Why do you think did Airbnb, LinkedIn and many more choose Node in production for high traffic apps?

Don't get me wrong, Python, Erlang, Haskell are great languages, I love them all but since V8/CrankshaftJIT/Node, JS is playing in a different league with an amazing cost-benefit ratio and I am confused by people ignoring this (however, I see rather Go and Clojure as real contenders to JS since they offer modern language concepts too paired with speed and real concurrency but they are again compiled and deploying JVM based stuff (Clojure) is no fun at all).

> deploying JVM based stuff (Clojure) is no fun at all).

In what world is typing "lein uberjar" and moving the resulting file to your webserver considered hard?

I would rather be out on the street busking.

While JS is quite flawed, it's nowhere near as bad as PHP. If JS kills all usage of PHP, the world will be a better place.

I actually don't think so, not that I have downvoted you. V8 was such a breakthrough at the time, it blew off the competition. Since then, many competitors have picked up the tricks and as shown in the bennchmark, uWSGI holds its own against Node/V8. As a result, the Node/V8 combination is not going to be so dominant in a few years in terms of performance. Another factors will come into play such as the language and the libraries available (the ecosystem). I would find it much harder to predict the dominance of the Noode/V8 ecosystem compared to that of Python or Java in a few years' time.

I think this is it exactly. The pace at which infrastructure, support and tools in "competing" ecosystems can be improved will outpace the rate at which people's miss can be changed about JS. There are 10-15 years of the language's, most of which were a Very Bad time for it's reputation, and only recently has the community of developers begun to land on solid development principles -- and you can hardly call those universal. I think most people fear callback spaghetti when they think of diving into someone else's JS code.

V8 and PyPy are similarly fast runtimes, but this benchmark tests libraries against each other. Not particularly relevant, especially as at least two of them are known to have slow IO.

As good as pypy is it's no replacement for python yet.

http://stackoverflow.com/questions/12867263/why-wasnt-pypy-i...

Reply on news.ycombinator.com