Proposal to Merge Pyston with Cpython | Hacker News Reader

Proposal to Merge Pyston with Cpython

discuss.python.org

138 pointscglace42 comments

This is so awesome.

My dream is to understand JIT compiler tech. Does anybody have any resources?

I've read the documentation of Pypy but I don't understand everything. And I've looked at dynasm for luajit.

I also read about Ruby's YJIT.

Do JIT compilers create machine code templates of bytecode instructions and then emit that? Or do they create basic blocks and then emit that as machine code?

My amd64 compiler is toy barebones that can evaluate ADD and MUL expressions https://github.com/samsquire/compiler I would love for it to become a JIT compiler!

Depends like everything in computing.

Clever ones are even multiple layers, with cache between runs and PGO metadata, or use a JIT server even.

Java and .NET runtimes across various implementations is where you will find these techniques.

> My dream is to understand JIT compiler tech. Does anybody have any resources?

Mike Pall's posts and LuaJIT is probably a good bet.

For a higher-level overview, I dimly remember that Mozilla used to have a number of posts on the subject. V8 / Chrome as well, especially as they moved back, forth, and sideways on the subkect.

> Do JIT compilers create machine code templates of bytecode instructions and then emit that? Or do they create basic blocks and then emit that as machine code?

That very much depends on the JIT. It can go from just swapping in precompiled code templates to type-directed compilation and AOT-style optimisations (loop unrolling, inlining, devirtualisation, ...), as well as things AOT compilers don't usually do (to my knowledge) like runtime-dispatched specialisation.

I haven't had a chance to dig into it yet, but I assumed tenderjit would probably be a good place to start…

I'm no expert, but you might be interested in: https://llvm.org/docs/tutorial/

There's also a Haskell version if you'd prefer: https://www.stephendiehl.com/llvm/

Idk how to do this in python as I'm not really good with it, but in C, to make your compiler a JIT, you would `mmap` a region as writeable, write the machine code to it that you already know how to generate, `mprotect` it as PROT_EXEC instead of PROC_WRITE, cast the pointer to the region to a function pointer, and then call it. These functions may be available in the python sys package but I don't really know.

I've implemented a "JIT" that takes machine code as hex and does this. Warning: it's complete garbage with no error checking but is a good proof of concept. https://gist.github.com/martinjacobd/3ee56f3c7b7ce621034ec3e...

There was a post on hn about this:

https://news.ycombinator.com/item?id=29566249

>There is also room for more optimizations

I surely hope so, else the "10%" speedup mention in the existing jit version is not worth it. Short of 2-3x in gneral, it wouldn't...

Think that is an average, it is so workload dependent. For some things it can be multiple times faster.

Still, it would need to have a better average. Kind of like how V8 or LuaJit has quite faster average speedup than CPython.

A lot of speed critical python especially in the stdlib is already written in C; a JIT will never speed it up. Other code is so dynamic that even an effort comparable to V8 would not help. But if the is some code left which can run twice as fast just because CPython has twice as many lines of code we may need to do that.

>Other code is so dynamic that even an effort comparable to V8 would not help

I hear that a lot, but I find it's bogus. Javascript (especially modern one, after ES6) is extremely dynamic too.

And a huge majority of Python code doesn't use much crazy dynamic features - a JIT can detect this and do its magic (V8s Jits detect such "could be dynamic but is static" code with several tricks and optimizes it).

Fully agree. The python community just wouldn't admit they lack someone as good as Mike Pall or the V8 team. The python performance in the computer language benchmark games is so embarrassing especially on the benchmarks not requiring external libraries [1].

[1] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Python is absurdly dynamic, but a JIT normally runs a profiler and discovers which paths are actually taken. If an instance attribute is always directly accessed, and there is no MRO lookup, descriptor / property lookup, getattr lookup, etc, the JIT can optimize it down to the trivial path.

If a non-trivial path is rarely taken, it can be run by interpreting the plain bytecode.

At least it's how I understand things like V8 work.

Smalltalk is also highly dynamic. But it was running so much faster than Java that the Smalltalk's JIT was adopted to speed up Java bytecode. By now Java may rival C speeds with competently written code.

If you need that, just load up the benchmark with tests that leverage the operations that the JIT speeds up the most.

Like everyone here is saying, so much of this is workload dependent, so you can't compare to V8 or LuaJit without controlling for workload.

Can anyone that understands these things (i.e. not me, a meek python end user) explain this in simpler terms?

I checked th Pyston website, it essentially says it makes your code faster and nothing about what's the catch and how it magically works.

A JIT is a just-in-time compiler. As your python code is running, the jit looks for opportunities to take little parts of that python code and compile them into executable machine code. Those then run much faster because they don't have to be interpreted instruction by instruction by the python runtime.

Not all of the code gets transformed this way so the speedup is modest.

Pyston is JIT compiler, which translates frequently executed bytecode into native code. JIT compilers are what makes Java and JavaScript fast. Pyston's case is closer to JavaScript, I suppose.

Since profiling and translation take time, it benefits mostly long-running code, especially servers, and is less useful for transient utilities.

Compiling takes time, so there is a built in tradeoff to doing a JIT compilation, if you compile the wrong thing the code slows down. In JavaScript at least code needs to be monomorphic to be compiled so if something is almost always called with one type, but occasionally another you can get a massive slowdown (at least in the past I'm assuming this is still the case).

More generally all the profiling and whatnot is much more complicated and so everything that entails.

I think the pros/cons of JIT compilation are pretty well known by now, and there's enough expertise floating around that they'd probably be able to implement it in a way that doesn't compromise performance. If I recall one of reasons it wasn't pursued in CPython was that Guido wanted it to be kept relatively simple, and JIT compilation might require some fairly complex additions to CPython which is otherwise relatively simple and easy-to-understand

My hope is that the Python team can keep it snappy by default so we don't end up with another Julia. And don't get me wrong, Julia is cool, but the primary drawback is its warmup time. And besides the adoption momentum that Python has, this is the main differentiator in my mind.

But Julia's warmup time should serve as a warning: the expertise you claim exists has not saved major projects from the drawbacks of JIT.

Julia's JIT is not typical. It compiles everything, not doing it piecemeal and smartly with different "execution" layers (like, say v8).

It's also based on LLVM, which historically has been slow and crap for JIT work.

There are Jitted languages that run circles on top of Python on running time and even startup time.

>In JavaScript at least code needs to be monomorphic to be compiled so if something is almost always called with one type, but occasionally another you can get a massive slowdown (at least in the past I'm assuming this is still the case).

Iirc, V8 has several layers of smarts to deal with this.

It compiles your program piecemeal as it runs.

The compiled parts don't need to run line by line through the interpreter loop anymore: they're already in memory in cpu-executable form.

And (don't know if it does) could even create dedicated faster code for parts of your code, if it can guess the types it works with.

Let's assume you speak English - and more importantly, not Chinese.

Now, you need to have a conversation with someone who only speaks Chinese. How do you do that? Well, you use an interpreter. They listen to what you say, and translate it to Chinese and speak on your behalf.

If you never had any interest in learning Chinese, this would be sufficient. Your conversation rate would be slow but it would get the job done.

Let's say there's an intense line of yes-or-no questioning, though. You need to say "yes" or "no" often, and decide that maybe learning the equivalent Chinese words or phrases would cut out the middleman, thus speeding up conversation in some cases.

This is what a JIT does, just replace English with Python's bytecode, and Chinese with your host CPU's architecture (e.g. x86 or ARM). The "when" of JITing code varies across implementations and approaches but the concept remains the same. Oftentimes the JIT runtime will translate just the hot codepaths.

If a compiler - e.g. one that compiles C code - is "ahead of time", then a runtime that compiles on the fly does so "just in time".

It looks like faster-cpython wants to include a "simple" (non tracing) JIT into 3.12[^1] anyway so maybe this is just what they need? Timing might be an issue though; since 3.11 was released 10/2022, CPython devs may already be hard at work on their own JIT impl.

[^1]: https://github.com/markshannon/faster-cpython/blob/master/pl...

I adore Pyston. It was literally a 2x speedup on my codebase, which as a free drop-in win, is pretty close to magic.

I wouldn't mind a bit more performance, but I don't know how this fits in with the current plans around speeding up the CPython interpreter.

Any thoughts on how likely Pyston is to be merged? Is this likely to be shot down?

CPython is very focused on speed; and is already planning and working on adding a JIT in the near term. I would guess they will implement their own rather then merge Pyston but the end result for people using Python should be similar.

The CPython team is (correctly) very concerned with code quality and the ease of maintenance. This favors starting with a very simple JIT; very aligned with the interpreter.

Why not pypy ?

This is a question to pypy folks.

The initiative in the message posted comes from Pyston folks themselves. They consider their code ready for that.

Not affiliated with either project: PyPy is a completely separate Python implementation without perfect ecosystem compatibility. Pyston is a small fork of CPython that replaces/adds on a couple of areas. They're very different projects, and Pyston is the only one that could be meaningfully merged into CPython rather than replacing it outright.

IMHO Maxime's YJIT ported to python would be the better way. But someone has to do it, and this just Python, not Ruby.

Php's compiler and jit would also be better, and much smaller.

For anyone curious, here's a set of benchmarks that compare Pyston vs PyPy (vs Python 3.11):

https://www.phoronix.com/review/python311-pyston-pypy/2

They're neck and neck in most benchmarks, interestingly enough.

As to why not PyPy ... well, PyPy is a complete reimagining of how Python is implemented, so a merge would be infeasible. It would have to be a very disruptive transition and the appetite in the Python world for a big transition is ... minimal.

PyPy is also slower when calling into C extensions, so that's one downside. I'm not familiar enough with it to speak thoroughly, though.

PyPy uses a modified Mark and Sweep garbage collector, CPython uses Reference Counting. C extensions such as NumPy (and so Pandas, sklearn etc) are compiled expecting Reference Counting. A translation layer is needed for memory management from PyPy to extensions like NumPy and that introduces overhead (historically - a lot of overhead).

So, theoretically, all C extensions could be rewritten, too.

Unless something changed pretty recently, pypy's database support is pretty weak. It's not a drop-in replacement for Cpython.

Because PyPy has never been a viable replacement for CPython engine in the confines of the CPython project.

For one, it would mean you'd have to throw away almost all CPython code.

Second, all the new code would be from totally different people than the CPython dev team (not very enticing).

Third, it would have compatibility issues with extensions and such.

Fourth, it would be slow calling into existing C code, which is like one of the major selling Pythons of Python (think pandas, numy, scipy, pytorch, and tons of others).

This would be great.

The lack of JIT love on the reference implementation is what makes me only use Python for UNIX scripting.

Proposal to replace Python with Nim :-p

Whats really amazing is that Python, with its rather large userbase, hasnt already picked some of this low hanging fruit.

Anyway I have been using pyston-autoloader-lite system wide, and so far the only thing it breaks is (unsurprisingly) Pytorch's torch.compile feature.

>Whats really amazing is that Python, with its rather large userbase, hasnt already picked some of this low hanging fruit.

Is it that amazing?

Adding a JIT to such a widely used language and maintaining compatibility (including for C extensions and such) is hardly a "low hanging fruit".

And it was tried time and again, by all kinds of different CPython forks (and also ports like PyPy).

Plus, "large userbase" doesn't translate to paid developers to devote time to this. Javascript needed full teams in Apple, Google, and Microsoft to make 3 JIT-ed engines after 10 years or so of not having any (and another 15 years to optimize them, which is still ongoing).

Reply on news.ycombinator.com