Hacker News Reader

Comment by harvie | Hacker News Reader

Unfortunately stuff like this never makes it to the upstream. And i am afraid to ask why. We had pypy for years, but never got merged with python. That is why there are still minor incompatibilities between pypy and "The Python", so it's not that useful as it might have been if it got merged with cpython at some point.

View on HN

I got a massive jump in performance when moving from Python 3.8 to 3.10 (over some function call optimizations I think, based on the project). And 3.11 got even better (up to 50% faster on special cases, and 10~15% on average) with respect to 3.10. Python 3.12 is already getting even more speedups and a there's a lot more down the road[0].

But Python core developers value keeping "not breaking anyones code" (Python 3 itself was a huge trip on that aspect and they're not making that mistake again), that's why things may seem slow on their end. But work is being done, and the results are there if you benchmark things.

[0] See https://github.com/faster-cpython/ideas/blob/main/FasterCPyt... however that's over a year old already and I'm sure I've read/heard more specifics

same. for my work project, 3.11 was performance-equivalent to pypy 3.9

Interesting. What is the status of the GIL these days?

Exactly the same as it ever was, but if you're doing cpu-heavy stuff on python objects you're doing it wrong because they'll never be Fast.

I think 3.12 will get subinterpreters, which will allow you to have multiple interpreters, each on its own thread, sharing the process memory space. So kind of like a stopgap in between having real multithreading and pythons current situation. I'm not sure what to think of it so far, so until there's some beta or RC build to test, I don't think I'll be able to form an opinion.

There is a branch of the 3.9 release that removed the GIL created by Sam Gross that you can read about here: https://gavincyi.github.io/2022-10-03-does-sam-gross-nogil-c...

There is some work to bring it up to 3.12 and some resistance to merge it into 3.X because of the impact on extension modules (they all have to be recompiled and in some cases changed a bit).

If you are interested in it, reach out to Sam. He has done a pretty impressive piece of engineering work.

I got performance regression going from 3.10 to 3.11

https://ltworf.github.io/typedload/performance.html

While it was still on RC builds I tested it on a few projects I usually run and I got between 0 and 15% faster wrt 3.10, without any regression. But sure, every project is different and it looks like you've bumped into some with your project.

If it's not too big, you might as well just leave things as they are and wait for 3.12 to see what changes then. Apparently the changes will be bigger, so it could be be much better than 3.11 (without being much worse, hopefully!).

Python 3.11 broke a lot of stuff in Debian, as did earlier versions of Python 3:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=python3.9&...

It's not like you can just "merge" pypy into python, they are totally different implementations. CPython is written in C and PyPy is written in RPython which is a subset of the python language that gets compiled, into into an interpreter with JIT support. You can actually write an interpreter for any language using RPython and their toolset, for example Ruby https://github.com/topazproject/topaz

Morever, a wonderful aspect of standard CPython is that you can compile it from source on a huge range of architectures in less than 5 minutes. Building Pypy from source is more difficult, and Pypy is significantly less portable (e.g., there is no viable WebAssembly version of Pypy).

No need to be afraid. The Python C extension API makes it very hard to make a JIT work well because of how it is implemented. C extensions are also part of why Python is so popular in the first place. If everybody wrote pure Python (like they write pure JavaScript), then the reference implementation would probably look like Pypy.

Reply on news.ycombinator.com