Hacker News Reader

Comment by pekk | Hacker News Reader

It isn't strange, it's standard practice. What would be strange is to force Python to hold one hand behind its back and be used in a totally unrealistic way that doesn't reflect normal practice. And by strange I mean basically dishonest about the performance available when using Python.

View on HN

Well, yes and no.

Python apps CAN be fast. Python itself isn't. It's dishonest to do a NumPy benchmark to show how "Python" is fast if the article implies that all code written in pure Python will be as fast. Then newbies will come, try to write fast algorithms and find out, they have to rewrite them in a faster language to get that advertised performance.

If someone wants to do the kinds of computations done by NumPy, what's relevant is how fast that actual practice is. Not some stupid alternative implementation intended to show that Python is slow, but which nobody would ever actually use.

In other words, it's nonsense to discard Python with NumPy as an option for fast computations just because we can imagine an imaginary world where NumPy was written differently, purely to penalize this environment for having used more than one language.

To be honest, we should benchmark what we actually have, not imaginary made-up handicapped versions of the things we are judging.

> the performance available when using Python

The honest thing would be to say "yes, Python is slow, but you can use some fast modules written in C".

You can even go farther and say that if you're going to use Python for non-trivial projects you better learn C (or some C generator like Cython).

The ubiquitous implementation of Python is written in C. So, for example, the list-sorting algorithm is written in C. If you want to eliminate C from the equation completely, I guess we now have to rewrite the C sort algorithm to be slower to show off "how slow Python really is" in isolation from C. But this is a benchmark of a fantasy world. That Python isn't the real Python we use. In the real world Python is not in isolation from C, by design, and this works well.

No. The language in which the interpreter is written is not the point here. The point is the interpretation overhead of the best (or "only" if we're being pedantic since Python is de facto whatever CPython does) interpreter.

That overhead is the price you pay for using Python so it's what matters when you compare languages for CPU bound projects.

It's a little strange to insist that CPython is the only Python interpreter and then reject all the C libraries that you can access for performance (because those C libraries are the big loss when shifting from CPython 2.7 to Pypy).

I think interpreter overhead and language ecosystem performance are probably best looked at as separate questions.

You only need to know C to get fast Python code for a tiny minority of situations. For one, most interesting C wrappers have already been written and second, the majority of C code you will use in normal business software contexts isn't used by FFI.

For a normal web application the request flow goes something like: nginx (WSGI) -> uwsgi -> web application code -> psycopg2 driver -> postgres. Only the web application part is written in Python, so for practical purposes you actually have a C stack that uses Python for business logic. Loads of libraries, from json parsers to templating libraries include optional C code for speedups.

Guess where the bottleneck is when you need to generate a dynamic page for each and every request.

In the database? About 90% of bottlenecks boil down to something along the lines of "this query is slow" or "there are a lot of queries here". This has been true since the dotcom boom

Guess again. All these benchmarks use the same database: http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...

Those are fairly uselss synthetic benchmarks. How many apps do you have in production that are purely key value stores with a random access profile?

And if you really have these performance characteristics (hint: it's unlikely), maybe it's time to get a edge caching CDN? Cutting 400ms of latency as you cross from Amsterdam to California is going to be the easiest performance boost you ever got

The point of these benchmarks is that, given the same database and the same queries, languages have different performance profiles.

If I build an application in Java and Python, with the same database backend, running the exact same set of SQL queries, the Java application should perform better.

The point is that with a db which hits the disk, the different execution times will generally look like rounding errors.

If you look at those benchmarks, for the same db queries on the same db backend, throughput and latency can differ by an order of magnitude. We're talking the difference between 200 or 2000 requests per second, or 50ms vs 500ms latency. That's not a rounding error.

My (and others') point is this obvious thumbs rule: If you hit the disk, that I/O will dominate the latencies. Even while using a scripting language.

E.g. a DB that fits in RAM is different, of course. It is hence easy to create benchmarks that generates every possible result.

So is your comment relevant to my point? Is there something about that benchmark which contradict the previous thumbs rule?

(And yes... With lots of calculations, scripting languages aren't a good idea either in many cases. Also obvious.)

The real answer is "it depends".

The whole rise of memcached and NoSQL should pretty clearly indicate that many developers are finding their database to be the bottleneck.

There's much less of a push for high performance languages, even though there are many that are also quite nice to work with (eg, Clojure). Since this is a Python discussion, searching for "Django PyPy" and "Django NoSQL" should be instructive.

You're combining a false dichotomy with snark, which really shouldn't have a place here on HN.

Well, at least some of the NoSQL movement was some people preferred to model data as documents or dictionaries or graph, etc instead of relations

In general -- the bottleneck for web servers are databases. That is why scripting languages (Perl, PHP, Python, Ruby, etc) are used for web pages.

If processing speed is needed... well, note that databases are seldom written in scripting languages.

Reply on news.ycombinator.com