It's written in Python and works fine to process thousands of jobs per day. Once you start having tens of thousands of jobs in the queue, it gets slow enough that it can back things up. This compounds the problem, eventually resulting in the whole thing crashing.
By switching the interpreter to PyPy, I was able to keep the data pipeline running at that scale without having to rewrite anything.
Haven't you just answered your own question? We know that people want to run CPU bound tasks in Python so much that they went to the effort of writing native modules because they couldn't do it in Python.
> python's lackluster concurrency support
This is a common misconception - Python actually has fully concurrent threads already.
This isn't true - they are fully concurrent.
The GIL prevents parallelism - not concurrency.
When a Python thread is holding on to the GIL (running Python bytecode), how many other Python threads can concurrently run in the same process?
The answer is zero.
Sure the interpreter releases the GIL every n bytecode ops, and C extensions can release the GIL before doing anything IO bound and reacquire it (i.e wait for it) afterwards, but that isn’t full concurrency, in my books.
You've been tricked by jargon. It's a common misconception. In this context, "concurrency" has a specific meaning that is different from the everyday one you're using in this sentence.
For a good introduction to what the two words mean in a software engineering context, check out this written version of Rob Pike's talk, "Concurrency is not Parallelism."
https://rakhim.org/summary-of-concurrency-is-not-parallellis...
The very tl;dr summary is:
> Concurrency is composition of independently executing things. . . Parallelism is simultaneous execution of multiple things.
When Python's threading model was implemented, parallelism just wasn't much of a concern. CPUs had a single core and could therefore only be working on one thing at a time. (In a macro sense; pipelining and supersclar architectures were still a thing, but not super relevant here.) Multithreading was not a way to do multiple things at once, it was just a way to ensure that some long-running calculation would not cause the program to lock up by, e.g., preventing it form responding to event queues in a timely manner. This was done by, not by running things in parallel, but by switching back and forth among them them very quickly.
Python's GIL was designed for this kind of situation. It's there to ensure that nothing bad happens if one of those context switches happens in the middle of a sensitive operation. Which is, strictly speaking, a concurrency concern and not a parallelism concern.
(It's also possible to have parallel work that is not concurrent, in which case locks are not necessary. But just because it's common for parallelism and concurrency to co-occur does not mean that they are the same thing.)
There are many sites and services where a rewrite in a new language is just not viable, and I still would recommend Python-everywhere to startups doing things remotely associated with data. So PyPy would be a tide that would lift many boats.
We had so much less baseline load afterwards that we were able to scale down a bunch of instances. The transition only took a few hours of effort fixing one or two incompatible dependencies, so it paid for itself in savings quickly, especially vs an approach of trying to rewrite the slowest bits in a faster language.
0. Because you don't have time to deal with the mess that C++ has become, and the amount of please-repeat-yourself-a-million-times crap you have to deal with (cmake files, header files, they give us a goddamn spaceship operator but not basic necessities like string split/join methods)
1. There are many use cases where faster execution time is nice to have, e.g. when results can be cached for a long time, or if it's a one-off data analysis script, but human time is far more expensive. If it costs $1000 more in engineer hours to write C++ instead of Python for that script that's only going to be run 10 times, that isn't a worthwhile tradeoff. Hell you could buy a new GPU for that money.
2. Because the same exact file can be deployed on arm32, arm64, and x86
3. Because CPU-intensive stuff is largely already optimized by numpy, numba, tensorflow, pytorch, etc.
Other languages don't have that same cycle, and Python has a LARGE amount of freely available packages that are able to help launch even quicker by not having to necessarily write everything oneself.
So at that point it comes down to what are you fastest in?
Strictly speaking, "CPU bound" is not an adjective that can be used to describe tasks. It's one that describes a particular program solving a particular task under a particular configuration. I've done no analysis on this myself, but I would be more than willing to believe that a CPU-bound job might be only a CPython-to-MyPy's worth of speedup away from instead being memory- or disk-bound.
It can be hard to tell the difference, too, since being stalled out while waiting on the memory controller shows up as CPU activity in htop.
If you are after deep profiling, you should definitely give it a try. My recollection is totally positive.
[0] https://software.intel.com/content/www/us/en/develop/tools/o...
For me this always just consists of reading a bunch of files in and then doing some basic aggregation on them. Years ago I benchmarked Python vs. PyPy and found no real benefit to it.
Here is the link to that if you'd like to read it: