Making Python fast – Adventures with mypyc
blog.meadsteve.dev
blog.meadsteve.dev
I am even rethinking that now because I was able to write a program in Go with an HTTP API and using JSON as the usual API interchange format in one night (all stdlib too), and it was so easy that I plan to pitch using it for several services we need to rewrite at work that are currently in Python. That would be very similar to what I wrote in a day.
If Python doesn’t fix their packaging, performance, and the massive expansion in the language, I think it’s going to start losing ground to other languages.
This is miles ahead of having to struggle with getting types correct in C++ before you even have anything resembling a workable solution.
You might have forgotten, but you had head scratchers too when you where learning. Everyone has them. I’ve taught people who are absolute geniuses, even they struggled initially. And sure you get over it, just like people can become quite adapt at programming in esoteric languages like brainfuck, but that doesn’t mean there aren’t any better ways.
(Disclosure: I'm a very minor volunteer contributor to that effort. I have a series of pull requests in the works that will improve runtime performance by about 1% across the board. I also have some more specialised improvements to memory.)
Do you know if there are any requirements for which modules can be compiled? E.g. can they be imported in other modules or do they have to be a leaf in the import tree/graph ?
So my reading is that it should be pretty seamless.
The blog post wanders around a little because I had to add setuptools and wheel building as my project had previously skipped this.
All other Python options I've seen feel too involved or leak too much into your code. Lagom seems to balance everything just right.
Thank you!
From their docs:
You can write computationally intensive tasks in Python while obeying a few extra rules imposed by Taichi to take advantage of the latter's high performance. Use decorators @ti.func and @ti.kernel as signals for Taichi to take over the implementation of the tasks, and Taichi's just-in-time (JIT) compiler would compile the decorated functions to machine code. All subsequent calls to them are executed on multi-CPU cores or GPUs. In a typical compute-intensive scenario (such as a numerical simulation), Taichi can lead to a 50x~100x speed up over native Python code.
Taichi's built-in ahead-of-time (AOT) system also allows you to export your code as binary/shader files, which can then be invoked in C/C++ and run without the Python environment.
That's cool af.
python
numpy: ~3.30ms
numpy with numba: ~2.90ms
node
tfjs: ~1.00ms
gpu.js: ~4.00ms
ndarray: ~118.00ms
vanilla loop: ~138.00ms
mathjs: ~1876.00ms
browser
tfjs webgpu: ~.16ms
tfjs webgl: ~.76ms
tfjs wasm: ~2.51ms
gpu.js: ~6.00ms
tfjs cpu: ~244.65ms
mathjs: ~3469.00ms
c
accelerate.h: ~.06ms
[0] https://cupy.dev/ [1] https://pytorch.org/ [2] https://www.tensorflow.org/
Would love to buy you a coffee and hear about your experience and the challenges and benefits.
Doubling performance is nice, though it does still leave a lot of performance on the table. I’d be curious to see a comparison between this and Cython.
Can someone explain more about how mypyc is in a better position to produce better optimizations than pypy, or am I confused about this?
But if mypyc has no runtime information to go on (which pypy does have), then certainly having some type information is better than having none.
Still nice, but not like golang or rust where you have a stand alone solution.
It's an alternative to nuitka, which I recommend to try out.
I mean regardless of whether mypy was going to make my code run faster I would have used it for the shear confidence it gives wrt to my code correctness. The fact that I can use that same code (untouched) to speed it up... that's just means I get to have my cake and eat it too :P
Many pieces of code run only a handful times but potentially move a lot of data. Sometimes reimplementing existing code in C/Haskell/Rust, including finding equivalent libraries, writing tests, and debugging, only because the computation turned out to be heavier than I had expected is not a good use of time. If that’s the place where I’m at, PyPy, mypyc, etc., might just do the trick.
But there is a disconnect between Python programming simplicity and Python speed, which stems from the fact that under the hood Python is doing much more than its minimal asm 'spiritual equivalent'.
But in a pure abstract theory sense, it shouldn't "need" to. I don't really care about the intricacies of garbage collection or global interpreter lock or page misses etc - what if I just care about "can I make this nice idea into reality in 10 minutes". The reality is that I'm just barely working with the tip of an iceberg composed of 60+ years of computer abstractions. But who can blame me - I am but a mere mortal.
If we could have a programming language that is both simple for the programmer and simple for the computer, it would be great. It's not that unreasonable that people start from the user experience side - making a simple language faster, by getting rid of unnecessary work - rather than the opposite extreme: making it simpler to come up with optimal machine code whose simplicity withstand contact with hundreds of vestigial appendages that just have to be dealt w bc of computer history (spanning from how to do a syscall to how to make a gui in some particular os)
It just makes me sad that in a world with multiple high-performance JIT engines (including pypy, for Python itself), the standard Python version that most people use is an interpreter. I know it's largely due to compatibility reasons (C extensions being deeply intertwined with CPython's API).
There is a really important (if not "magic") difference between JavaScript and Python. JS has always (well, since IE added support) been a language with multiple widely-used implementations in the wild, which has prevented the emergence of a third-party package ecosystem which is heavily tied to one particular implementation. Python on the other hand is for a large proportion of the userbase considered CPython, with alternate implementations being second class citizens, despite some truly impressive efforts on the latter.
The fact that packages written in JS are not tied to (or at least work best with) a single implementation is also what made it possible for developers of JS engines to experiment with different implementation approaches, including JIT. While I'm not intimately familiar with writing native extension modules for Node (having dabbled only a little), my understanding is the API surface is much narrower than Python, allowing for changes in the engine that avoid breaking APIs. But there is less need for native modules in JS, because of the presence of JIT in all major engines.
this is misleading, if one sees the phrase "interpreter" as that code is represented as syntax-derived trees or other datastructures which are then traversed at runtime to produce results - someone correct me if I'm wrong but this would apply to well known interpreted languages like Perl 5. cPython is a bytecode interpreter, not conceptually unlike the Java VM before JITs were added. It just happens to compile scripts to bytecode on the fly.
Almost all execution techniques include some combination of compilation and interpretation. Even some ASTs include aspects of transformation to construct them from the source code, which we could call a compiler. Native compilers sometimes have to interpret metadata to do things like roll forward for deoptimisation.
But most people in the field would describe CPython firmly as an 'interpreter'.
Yeah? I think almost everyone would?
> and you'd call Java an interpreted language?
Java is interpreted in many ways, and compiled in many ways, as I said it's complicated. It's compiled to bytecode, which is interpreted until it's time to be compiled... at which point it's abstract interpreted to a graph, which is compiled to machine code, until it needs to deoptimise at which point the metadata from the graph is interpreted again, allowing it to jump back into the original interpreter.
But if it didn't have the JIT it'd always be an interpreter running.
This is a common implementation approach - parse the source to generate an AST, transform the AST to bytecode, then interpret the bytecode. It's still interpretation, and is slow. Contrast to JIT engines which transform the intermediate code (whether that's AST or bytecode) to machine code, and is fast.
Perl uses the same execution method you describe for cPython.
(Not trying to interrogate you or prove you wrong, but I've got an interest in optimising very difficult meta-programming patterns.)
This means nothing is guaranteed and so every instruction must do multiple checks to make sure data structures are what is expected at the current moment.
This is true of JS as well, but to a lesser extent.
Aren't all the things you mentioned already fixed by deoptimisation?
You assume constants cannot be modified, and then get the code that wants to modify constants to do the work of stopping everyone who is assuming a constant value, and modify them that they need to pick up the new value?
> To deoptimize means to jump from more optimised code to less optimized code. In practice that usually means to jump from just-in-time compiled machine code back into an interpreter. If we can do this at any point, and if we can perfectly restore the entire state of the interpreter, then we can start to throw away those checks in our optimized code, and instead we can deoptimize when the check would fail.
https://chrisseaton.com/truffleruby/deoptimizing/
I work on a compiler for Ruby, and mutable constants and the ability to monkey patch etc adds literally zero extra checks to optimised code.
You can write a new compiler if you'd like, as detailed on this page. But CPython doesn't work that way and 99% of the ecosystem is targeted there.
There is some work on making more assumptions as it runs, now that the project has funding. This is about where my off-top-of-head knowledge ends however so someone else will want to chime in here. The HN search probably has a few blog posts and discussions as well.
Yeah that’s the point - the JIT takes that capitalisation as a hint to treat it as a true constant and bake the value in until it’s redefined.
This is all solved stuff and isn’t a barrier to implementing a powerful JIT for Python if someone wanted to.
I'll give you one you could have used - the GIL - however I'm not sure the GIL's semantics are really specified for Python, they're an implementation detail people accidentally have relied on.