How the CPython compiler works
tenthousandmeters.com
tenthousandmeters.com
The python bytecode interpreter mainloop can be found in ceval.c. There is a big switch case for each python bytecode opcode starting at https://github.com/python/cpython/blob/master/Python/ceval.c...
One can see even the simplest operations call quite a lot of stuff. I gave up on fast pure cpython after reading ceval.c...
The BEAM loads whole modules at a time, and as part of this process it means that instruction names in the bytecode are essentially replaced with the address of the code which interpret that instruction.
This means that once loading is complete there's no big switch statement to visit: the VM jumps directly to the part of its own code which interprets that instruction.
The BEAM loads bytecode into memory and then goes through the loaded code to directly thread it. It's not an inherent consequence of code loading in general.
https://en.wikipedia.org/wiki/Threaded_code#Direct_threading
Otherwise, Clang has access to the same optimizations that Numba has as they both share LLVM as their optimizing compiler. Beyond that, I think a fairer comparison is C w/ OpenMP vs Numba for parallel processing if syntactic brevity is the metric.
But as a general rule, the programmer will have a hard time "outsmarting" the compiler by using different objects or abstractions.
Compiler authors have spent many hundreds of combined hours making C code run as fast as possible, and optimizing the switch statement is something they've most certainly done to death.
I.e. if your control flow is switch-case-like, use a switch-case.
https://github.com/python/cpython/blob/master/Python/ceval.c...
If the case values are dense (such as just the integers from 0 to 9), the compiler will turn it into a jump table.
If too sparse for jump table, then typically will be turned into a binary search tree.
Of course, it doesn't mean there aren't companies who can't be successful in spite of these limitations--if you make enough money you can afford to throw money at these problems and make them better (you don't need efficient compute or parallelism if you can afford to pay a team to set up and operate a Spark cluster for you not to mention the cost of the additional VMs and the additional effort to write spark instead of a native language).
Nobody writes performance-critical code in pure python.
Not sure how "tooling" is bad, what would you say is limited there?
Package management, again, what package management problems are unique to Python? Many people say this but it seems the problems they bring up are not unique to pip or the python ecosystem, same problems are found with rubygems, npm, Maven, etc...
Maintainability is a responsibility of developers and not a programming language, and unmaintainable code can easily be written with any language. However I'd argue Python should score positive points for maintainability; one of the languages I feel most comfortable picking up old code from someone else and groking it easily.
One of Python's "design goals" is to integrate well with C by exposing every detail of the Python interpreter to C extensions. Since Python's performance is so abysmal, the ecosystem has come to lean heavily on C-extensions, and because the ecosystem leans so heavily on C-extensions, very few changes can be made to the CPython interpreter without breaking compatibility with the ecosystem--if we can't change the interpreter, we can't optimize it, and we're damned to a world in which Python is slow. Pypy is doing yeoman's work by building a new JITed interpreter that makes pure-Python code a lot faster, but it still has a lot of compatibility problems with important C packages. For instance, you can't talk to a Postgres database without using an obscure package that hasn't seen a commit in years.
> Nobody writes performance-critical code in pure python.
Not successfully, no. But if you buy the marketing, you'll be led to believe that Python has an answer to every performance problem. "Go ahead and start your project in Python. Don't worry about performance--if your program is too slow, you can just $X" where "X" is one of "rewrite the slow bits in C" or "use pandas" or "use multiprocessing". No one tells you that those options really fall over for a huge swath of real-world workloads, for the same reason: in many/most cases, the cost of de/serializing Python objects is greater than the savings from C or parallelism. It's only economical for those precious few cases where you can do a lot of consecutive work outside of Python.
> Not sure how "tooling" is bad, what would you say is limited there?
Precious little static analysis is available for Python, documentation generation options are generally bad (partly because they can't just generate the type information per the previous static analysis point, but also because they make a whole host of bad decisions, like putting everything on the same page and making you scroll around to figure out which class's __init__ method you're looking at presently) and you're still on the hook for operating the CI tools that generate and publish the documentation packages, tools tend to be written in Python and thus are really slow (e.g., formatters, package managers, etc), no static analysis means no dead code elimination and thus an enormous installed footprint well into the hundreds of megabytes (this problem is exacerbated by the weight of OOP in the Python ecosystem, which means everyone who wants a Book data structure depends on the whole universe of things that people do with Books), static distribution is still a joke--you end up bundling 250mb zip files and you still need to have the right version of Python and the right .so/.dlls installed on the target system, etc.
> Package management, again, what package management problems are unique to Python?
In order to figure out what a Python package's dependency tree looks like, you have to download the whole thing. This makes it difficult to have performant package managers (or rather those that are performant are unsafe because they splat things into the python environment and punt on making sure they don't have multiple versions of some transitive dependency). Python is also held hostage by an ecosystem of C-extensions, so its packages have to support the whole universe of terrible C package management decisions. Also, there is still no production-ready Python package manager that supports reproducible builds (i.e., respects lockfiles). I don't know about Ruby, but NPM, Go, and Rust don't have these issues and I'm pretty sure Java, C#, and Ruby don't either.
> Maintainability is a responsibility of developers and not a programming language, and unmaintainable code can easily be written with any language
It's a lot easier in a language without any rails to guide developers toward good development practices (by making it disproportionately harder to write hacky code), by which I mostly mean a static type system. Mypy is gaining traction, but it's still not in the same ballpark as other languages' type systems and it's moving at a snail's pace (no doubt other languages benefit from investment or else static typing was built into the original design, but those excuses don't make my team's code more maintainable). Not only is it rails to guide them on the right path, but it's also things like "type documentation is always correct" and "refactoring is easy so people actually do it".
Also, some of those $X are mutually exclusive.
If you write a lot of the performance-sensitive code in C; you're going to have complex C data structures. Now if you want parallelism, well too bad: you can't use multiprocessing, because you can't easily share your C data structure across multiple processes.
To actually use multiple cores without getting killed by the GIL, you end up having to replace a lot of Python code with C -- not just the most performance critical portion.
Copying a multi-GB data structure for each CPU core would take way too much memory, so we tried doing stuff with shared memory, but it's complicated. We spent months of developer time on this and still can't really scale beyond two cores, for something that would be embarrassingly parallel in any other programming language :(
The "mixing C and Python" solution is a trap. If performance might be important at any point in the future and you use Python; better plan for a complete rewrite in a different language.
--- On reproducible builds: given your examples from other languages, Python has such tools too e.g., pip-tools package.
I wouldn’t recommend pure Python (that is, without Numpy or Scipy) for machine learning, but it’s great in its own niches which are actually pretty enormous.
This is an understatement. Python is the slowest mainstream language. JS is about an order of magnitude faster. Go, C#, Java, etc are about 2-3 orders of magnitude faster. C, C++, and Rust are nearly 4 orders of magnitude faster. If you might ever care about performance at all, Python is limiting.
No doubt that if all you're ever doing is a little glue code that dispatches to a database (or to ML models or whatever), then Python is fine. The problems come about when you try to do more interesting things, especially involving a complex Python object model that doesn't map neatly to Postgres or Pandas. I've never worked on a Python project where this wasn't the case, and where the organization wasn't doing some desperate optimization (e.g., standing up a Spark cluster) to stave off a ridiculously gracious timeout limit (e.g., 60s for browser timeouts). Meanwhile, had the same server been written in Go, the naive implementation would have subsecond response times and a little bit of trivial optimization (the kinds of optimizations that aren't available in Python for a myriad of reasons) could get it down to double digit milliseconds.
Since we rarely know all of the types of bottlenecks a project will encounter in its years of service, Python has become something of a last resort for me (especially given all of the great options that exist nowadays). And none of this is to say anything about the tooling, distribution, or package management problems that plague Python either.
Right, but the difference is whether you will ever have to do more than "just a little glue" and making sure you can easily delegate to a different tool to handle those cases. In my experience, there are precious few cases where you never have to do more than just a little glue. Basically any application in which Python owns the data model (e.g., the data model is a non-trivial tree of Python class objects), this is incompatible.
> Sure, and we agree: you wouldn’t use pure Python for something where every scrap of performance matters.
I don't want to be disagreeable, but this is the sort of thing I would say about Go or Java--you shouldn't use it for cases where every scrap of performance matters. But there's an enormous gap between the things Python can do and the things that Go and Java can do. Moreover, if you do run into problems with Go or Java, there's a lot of optimization available to you (at least not in the general case, although there are niches where you can gain a lot of performance with pandas or similar).
Moreover, there's just not a good reason to choose it these days. There are better options without the traps.
There are also a lot of programming language who don't bother touching Python because the team doesn't seem to care about runtime performance as their responsibility. There's a lot of excellent talent out there, and they're mostly being denied and shot down.
Edit: Oh yea, there we go. You're using CSS zoom[0], which is non-standard.
A good JIT is hard to write, and harder to retrofit. It usually increases startup times, attack surface, and code complexity. Besides, Python already has a handful of JIT-based implementations, and I don't see how adding the reference implementation (which out to be simple!) to that list really improves anything.
2. It's a forum, not everything has to be a direct response.
3. Either way, this was a direct response: CPython does not have a JIT because there's little reason for it to have a JIT, and many good reasons why it shouldn't.
4. If you want a JIT'd Python so badly, why not use PyPy?
Unladen Swallow was a version of CPython that added JIT support. PEP-3146 describes an unsuccessful attempt to merge it into CPython [1]. Here's a postmortem [2]
Starting with version 3.6, CPython allows to set a custom frame evaluation function on the interpreter and to store JIT compiled code in a code object. This was done to support JIT in a form of a separate module and is described in PEP 523 [3].
This is very interesting topic. I should study it in detail and probably write a post. I hope I'll be able to give a more thorough answer someday.
[1] https://www.python.org/dev/peps/pep-3146
[2] http://qinsb.blogspot.com/2011/03/unladen-swallow-retrospect...
Intel has has a library that lets you use pandas in numba code, so it's I feel pretty confident using it in production without breaking cpython compatibility
It actually worked quite well, though was pretty memory hungry.