How slow is Python really? Or how fast is your language?
codegolf.stackexchange.com
codegolf.stackexchange.com
If your problem is numerical in nature, you can call popular C modules (numpy, etc) or write your own.
If your functions and data are pickleable, you can use multiprocessing but run into Amdahl's Law.
Maybe you try Celery / Gearman introducing IO bottlenecks transferring data to workers.
Otherwise you might end up with PyPy (poor CPython extension module support) and still restricted by the GIL. Or you'll try Cython, a bastard of C and Python.
Python has been my primary language the past few years and it's great for exploratory coding, prototypes, or smaller projects. However it's starting to lose some of the charm. Julia is filling in as a great substitute for scientific coding in a single language stack, and Go / Rust / Haskell for the other stuff. I've switched back to the static language camp after working in a multi-MLOC Python codebase.
I've been wondering about why so many python devs have migrated to using Go recently instead of Julia, given that Julia is a lot closer to python and has performed as good as, if not better than, Go in some benchmarks [1]. Granted I've really only toyed with Julia and Go a few times as I've never really needed the performance much myself, but I'm curious about your preference of Go/Rust over Julia for "the other stuff".
What would you say makes Julia less suitable (or Go more suitable) for nonscientific applications? Is it just the community/support aspect? Cause that seems like an easy tide to overturn by simply raising more awareness about it (we see Go/Rust/Haskell blog posts on the front page of HN every week, but not too many Julia posts).
Just curious cause I'm not nearly experienced enough with any of these young languages yet to know any better, and have only recently started to consider taking at least one of them up more seriously.
I'm yet to try Go, but Julia is great for me: it's like all the good parts of Python + extra speed. You get the interactivity, the good documentation, the community, the packages, and all bundled into something that's easy to use and gives you code that runs many times faster than native Python with no real extra effort.
I'm migrating an in house ORM to SQLAlchemy. Lack of compiler support and/or static code analysis makes the transition more difficult than it needs to be.
Dynamic typing allows one to defer error handling to the future, essentially creating technical debt for the sake of developer speed and convenience. For many use cases this is an acceptable trade off.
However as a codebase grows in complexity, it's better to handle errors as early as possible since the cost of fixing an error grows exponentially the farther it is from the developer (costs in ascending order: editor < compiler < testing < code review < production).
A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns.
Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary.
A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer.
A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation.
Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period.
If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules.
The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36.
from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002
Moreover, the benchmark you listed here are still mostly scientific computation related. Just from the benchmark, I'm not convinced enough that Julia beat Go from the perspective of a backend system, which is what I primarily use Go for.
Go and Julia really serves totally different purpose. Go want to replace Python in backend and the Python ecosystem matters little here. Julia want to replace Python in scientific computing and Python ecosystem matters a LOT here.
Are you sure you were using the language properly?
Some constructs are still slow, and others prevent type inference, which can also slow things down...
- It doesn't have Google / Thompson / Rsc / etc behind it
- It looks more like Ruby than C (say what you will about C-like syntax, but I
think the success of Java and C++ has proved that point)
Living CPU paycheck to paycheck and on occasion taking performance payday loans (breaking out to C) is not an efficient way to manage resources.
Sure, you can hire an extra C++ developer for 150k or just give your python developer a company credit card to use for extra AWS machines. I'm quite sure the second option is a lot cheaper.
When traffic and data ramp up, 150k for a guy to cut your capex by 10% sounds like a steal.
CPU Cycles vs Developers
This is a common false dichotomy: that expending more CPU cycles on the language runtime makes a language more efficient in terms of developer productivity than a language that expends fewer CPU cycles on its runtime.
The inherent truth of this statement isn't deductively obvious, however.
If you assume, for example, that dynamic languages are more productive for developers (I don't, but it's a common argument, so we'll go with it), then this is easily disproven simply by comparing the performance of a highly optimized JIT -- such as V8's -- against Python's interpreter.
Irrespective of the language, there exists differing levels of quality of runtime. V8 is faster than Python; this is simply because V8 is a better runtime JIT than Python is an interpreter.
If we discard the unproven "dynamic languages are more developer efficient" hypothesis, things become even more stark. The JVM, for example, with real threads, highly optimized JIT, and the advantages of operating on a much more well-typed system, is in fact faster than V8 -- all with a "managed" language, and not C++.
Taking that a step further, we've recently begun rediscovering ahead-of-time compilation of so called "managed" or "high-level" languages. The favoritism given towards JIT arose out of efforts to achieve high performance in dynamic languages were very little can be statically guaranteed. What has recently become clear is this: in more static languages, we can achieve the same level of "managed" runtime without introducing the overhead or complexity of JIT at all!
All combined, I see very little argument for a dichotomous choice between "inefficient, high level language" or "efficient, low level language" -- the choices seem to simply be "inefficient" vs "efficient".
Relative Value of High-Paid Developers
Lastly, I wish to address the "extra C++ developer for 150K". I'll keep this one brief -- simply put, I would hypothesize that a $150K expert-level engineer is worth anywhere from 2-10 $80K non-expert engineers.
This is simply due to an expert-level engineer's experience and deep knowledge of the technology stack allowing them to architect systems to achieve maximum maintenance, developer and system efficiency over time.
Computing is a value multiplier: the potential gains and losses of lower multipliers can be objectively enormous.
Having managed teams where I've inherited cheaper, more junior engineers, versus teams where I've hand-picked a small group of extremely experienced engineers, I've saved time, money, and headaches with the more expensive engineers every time.
However, I think there's another false dichotomy there: a $150K expert-level engineer versus $80K non-expert engineers. In reality, there are only expert and non-expert engineers---pay doesn't seem to be a particularly discriminator. Further, in all respects it is difficult to tell the difference between an expert and a non-expert, hence all the fun interviewing follies. And even the best of those don't work particularly well.
When it comes to differentiating expertise, we often have to look for secondary indicators; pay has an extremely poor correlation with competence, especially in high-demand job markets.
The learning curve for Scala may be steep though.
For example, hg is now faster than git. Git is written by great C hackers (including Linus no less), and yet hg is faster than git. See the facebook benchmarks for evidence.
Python has static checking of interfaces, and types if you want. It also has IDEs which can check a lot of things for you. It turns out that dynamic typed languages can be checked for quite a lot of things.
Check out using mmap based datastructures to do shared memory between workers.
You're comparing the runtimes of two different algorithms, so your results are inevitably meaningless.
> It turns out that dynamic typed languages can be checked for quite a lot of things.
And yet, statically typed languages can still be checked for a lot more things.
What is your evidence? Can you provide a citation?
Modern tools can statically check python for a lot of things.
Here are some things you can statically check with python:
* implements an interface
* unused names
* assigned but never used
That 'implements an interface' one is important. Since with interfaces you can specify things like return types, and type arguments.
There are also @param markers in doc strings, where you can specify argument and return types. These can be used by tools like IDEs (eg pycharm) and other static checkers.
Full program type inference is now doable for large parts of python. See the shedskin, and Rpython restricted subsets for two implementations.
>most of Mercurial is written in Python, with a small part in portable C for performance reasons.
It shows that Python can be fast enough if you leave the heavy computational parts to libraries written in other languages. Which is interesting, but doesn't say much about the speed of Python itself.
Also, python is not an implementation of python.
These benchmarks are always funny, because real systems use different components, yet the benchmarks stick to some fake, non-real-world way of measuring.
Oh, the garbage collection in java pauses for multiple seconds soemtimes... but it's not slow because we ignore that in our benchmarks. Oh, it's not fast the first time, because the jit hasn't warmed up? Let's ignore that in our benchmarks too. Um... yeah. Good one.
This benchmark is also flawed to since people would probably use numexpr in the real world. Which is much faster than plain numpy. So python would be even faster than they say.
Mercurial(hg) is now faster than git. Git is written by great C hackers (including Linus no less), and yet hg is faster than git. See the facebook benchmarks for evidence.
Using the right tool for the job can mean using multiple languages together for where they are best. Want clarity and performance? Then C/asm + python is an ok combination.
> optimization by precomputing F values
> I also optimize by skipping the rest of the convolution if the first result isn't zero.
If you don't measure implementations of the same algorithm, you hardly have a fair language/library benchmark.
Put another way, one of the benefits of Python (and drawbacks of using an external library) is that you have more control over the algorithm and exactly what it does.
That said, a sample size of one will hardly give you an accurate picture.
"Measuring implementations of [different] algorithms is how you benchmark algorithms"
Would you exclude use of stdlib parts that are written in C? Would you say Javascript can't run serverside cause Node.js is not part of your imagined "core language"? Or it doesn't say much about the speed of C when you use a better/more optimized compiler or compiler flags?
Python apps CAN be fast. Python itself isn't. It's dishonest to do a NumPy benchmark to show how "Python" is fast if the article implies that all code written in pure Python will be as fast. Then newbies will come, try to write fast algorithms and find out, they have to rewrite them in a faster language to get that advertised performance.
In other words, it's nonsense to discard Python with NumPy as an option for fast computations just because we can imagine an imaginary world where NumPy was written differently, purely to penalize this environment for having used more than one language.
To be honest, we should benchmark what we actually have, not imaginary made-up handicapped versions of the things we are judging.
The honest thing would be to say "yes, Python is slow, but you can use some fast modules written in C".
You can even go farther and say that if you're going to use Python for non-trivial projects you better learn C (or some C generator like Cython).
For a normal web application the request flow goes something like: nginx (WSGI) -> uwsgi -> web application code -> psycopg2 driver -> postgres. Only the web application part is written in Python, so for practical purposes you actually have a C stack that uses Python for business logic. Loads of libraries, from json parsers to templating libraries include optional C code for speedups.
The whole rise of memcached and NoSQL should pretty clearly indicate that many developers are finding their database to be the bottleneck.
There's much less of a push for high performance languages, even though there are many that are also quite nice to work with (eg, Clojure). Since this is a Python discussion, searching for "Django PyPy" and "Django NoSQL" should be instructive.
You're combining a false dichotomy with snark, which really shouldn't have a place here on HN.
And if you really have these performance characteristics (hint: it's unlikely), maybe it's time to get a edge caching CDN? Cutting 400ms of latency as you cross from Amsterdam to California is going to be the easiest performance boost you ever got
If I build an application in Java and Python, with the same database backend, running the exact same set of SQL queries, the Java application should perform better.
E.g. a DB that fits in RAM is different, of course. It is hence easy to create benchmarks that generates every possible result.
So is your comment relevant to my point? Is there something about that benchmark which contradict the previous thumbs rule?
(And yes... With lots of calculations, scripting languages aren't a good idea either in many cases. Also obvious.)
If processing speed is needed... well, note that databases are seldom written in scripting languages.
That overhead is the price you pay for using Python so it's what matters when you compare languages for CPU bound projects.
I think interpreter overhead and language ecosystem performance are probably best looked at as separate questions.
The average code-quality in rubygems is just not very high. Consequently most libraries are completely oblivious to performance aspects.
This reaches straight into the core infrastructure (rubygems, bundler) and all major projects (Rails). Leading to the simple fact that Ruby loses out on many practical benchmarks before your script has even loaded.
Likewise the synergies of less than bright behaviors from all the gems in your average Rails project (and no least Rails itself) do indeed make the performance gap towards an average django project much larger than the mere difference in sheer interpreter performance.
That's all not meant to bash Ruby anyway. It's a trade-off me and many others are willing to make, for the convenience that ruby provides after it has finally set its fat belly into motion.
But let's not pretend these differences don't exist when everyone who has ever used both languages knows them all to well.
However, even "real world" anecdotes in this area can be a minefield.
Take, for example, an existing Python application that's slow which requires a rewrite to fix fundamental architectural changes.
Because you feel you don't need necessarily need the flexibility of Python the second time around (as you've moved out of the experimental or exploratory phase of development), you decide to rewrite it in, say, Go, or D or $whatever.
The finished result turns out to be 100X faster—which is great!—but the danger is always there that you internalise or condense that as "lamby rewrote Python system X in Go and it was 100X faster!"
If my C is 1000x faster and saves me 60 seconds every time I run the program, but takes an extra 2 days to write initially, and the program is seeing lots of edits meaning that on average I have to wait 2 minutes for it to compile then I am MUCH better off with the slower MATLAB until I am running the same thing a few thousand times.
Plus there is the fact that I can look at HN while a slightly slower program is running, so I win both ways.
It's not hard to imagine a world where you instead use Haskell, prototyping your code in GHCi or even just writing it in Haskell directly, pay a minimal speed penalty for development since you're not being forced to use a klunky type system, and get compiled speeds or even GPGPU execution straight out of the box. (And before anyone freaks out about Haskell, using it for numeric computations requires pretty much zero knowledge about anything exotic... it's pretty straightforward.) It's not out of the question that using Haskell in this way would prototype even faster than a dynamic language, because when it gives you a type error at compile time rather than at runtime, or worse, running a nonsense computation that you only discover afterwards was nonsense, you could save a lot of time.
I don't think there has to be an inherent penalty to develop with native-speed tech... I think it's just how history went.
Exactly. I think that is correlated very well with single core CPU speedups.
Remember when Python was rising the fastest, single core CPU speed was also pretty much doubling every year. SMP machines were exotic beasts for most developers back then.
So just waiting for 2-3 years you got very nice speedup and Python ran correspondingly faster (and fast enough!).
Then we started to see multiple cores, hyperthreads, and so on. That is when talk about the GIL started. Before that nobody cared about the GIL much. But at some point, it was all GIL,GIL,GIL.
> It's not hard to imagine a world where you instead use Haskell
Hmm interesting. I wonder if that approach is ever taken in a curriculum. Teach kids to start with Haskell. It would be interesting.
"Hmm interesting. I wonder if that approach is ever taken in a curriculum. Teach kids to start with Haskell. It would be interesting."
To be clear, I was explicitly discussing the "heavy-duty numerical computation" case, where typing as strong as Haskell's isn't even that hard. Learn some magic incantations for loading and saving data, and it would be easy to concentrate on just the manipulations.
But yes, people have done this and anecdotally report significant success. The Google search "Haskell children" (no quotes in the real search) comes up with what I know about, so I'll include that in this post by reference. It provides support for the theory that Haskell is not that intrinsically hard, it's just so foreign to what people know. If you don't start out knowing anything, it's not that weird.
Also, 2 minutes per change to compile the object files affected and link the executable seems a bit excessive considering the entire Linux kernel can generally be built from scratch in less time than that (assuming a modern system).
spectral-norm Python 3 #5 program (no numpy)
spectral-norm Python 3 #3 program (numpy)
spectral-norm Python 3 #2 program (numpy + ignore separate functions requirement)
Deleted comment
Why would you use Numpy for arrays that small? Oh, looks like someone actually just wrote it in CPython, no Numpy, and it clocked in at 0.283s. Which is fine. It's Python.
This thread reminds me of the scene in RoboCop where Peter Weller gets shot to pieces. Peter Weller is Python and the criminals are the other languages.
Not that python is fast, it isn't. And using numpy seems a bit disingenuous anyways "Oh my python program is faster because I use a library that's 95% C"
If you enjoyed this Python optimization, you may also enjoy: http://stackoverflow.com/questions/17529342/
This sort of thing comes up a lot: people write mathematical code which is gratuitously inefficient, very often simply because they use a lot of loops, repeated computations, and improper data structures. So pretty much the same as any other language, plus the extra subtlety of knowing how and why to use NumPy (as it turned out, this was not a good time for it, though that was not obvious).
"How fast is the code produced by your compiler."
I keep seeing this misconception about languages vs implementations.
EDIT: Clarified what my original remark meant.
The "misconception" may be the casual assumption that the runtimes we have today are necessarily the optimal runtimes, which is not generally true. But after the past 5-10 years, in which enormous amounts of effort have been poured into salvaging our "dynamic" language's (Python, JS, etc.) run speeds, which has pretty much resulted in them flatlining around ~5 times slower than C with what strikes me as little realistic prospect of getting much lower than that, it's really getting time to admit that language design decisions do in fact impact the ultimate speed a language will be capable of running at. (For an example in the opposite direction, see LuaJIT, a "dynamic" language that due to careful design can often run at near-C.)
(BTW, before someone jumps in, no, current Javascript VMs do NOT run at speeds comparable to C. This is a common misconception. On trivial code that manipulates numbers only you can get a particular benchmark to run at C speeds, but no current JS VM runs at C speeds in general, nor really comes even close. That's why we need asm.js... if JS VMs were already at C speeds you wouldn't be able to get such speed improvements from asm.js.)
As for the current state of native compilers for dynamic languages, they suffer from the fact that past the Smalltalk/Lisp Machines days, very little focus has given to them in the industry.
Hence little money for research, while optimizing compilers for static typed languages where getting improved.
Dylan was the last dynamic language with a AOT compiler targeted to the industry as system programming language, before being canceled by Apple.
If it wasn't for the JavaScript JIT wars, probably the situation of compilers for dynamic languages would be much worse.
Once you add sophisticated compilation, dynamic languages implementations are no longer 'dynamic'.
This topic is pretty much solved for Lisp. On one extreme we have fully dynamic interpreters + then dynamic AOT compiler based ones. For delivery there are static delivery modes available (for example with treeshakers as in LispWorks).
On the extreme side you get full program compilers like Stalin (for a subset of Scheme) or like mocl (a recent compiler for a static subset of Common Lisp, for iOS and Android).
That's just a long way of agreeing with me. If one language can require a great deal more effort than another to run quickly (and I'm not assuming the slow language even makes it to the fast language's speed here), then it is therefore the case that languages designs do impact the experienced run time performance. That's the entire point.
And given where the plateau on the dynamic languages seems to be getting drawn, I see no reason to even hope that a dynamic language will ever run as fast as C in general. They are plateauing way too high for that to be the case, probably even with infinite investment on current hardware.
As for Go, compilers for languages with modules can run circles around C and C++ toolchains until they had proper support for them.
Different things use different types of randomness. Some are fast. Some are slow. If your comparison is not using the same type of randomness, that comparison is comparatively useless.