Hacker News Reader

Comment by pjmlp | Hacker News Reader

I really find a bummer that there is such a resistance to make JIT enable versions available for download, alongside the other variants.

Naturally I can easily compile my own Python 3.13 version, no biggie.

However from my experience, this makes many people that could potentially try it out and give feedback, don't care and rather wait.

View on HN

JIT is in weird place now because according to a recent talk from PyCON US by it’s author, the tier 2 optimizer that prepares code for JIT reduces performance by 20% and JIT just recovers that performance loss.

A lot of langugage is still not optimized by tier 2 [1] and even less has copy and patch templates for JIT. And JIT itself currently has some memory management issues to iron out.

[1]: https://github.com/python/cpython/issues/118093

Talk by Brandt Butcher was there but it was made private:

https://www.youtube.com/watch?v=wr0fVU3Ajwc

I have seen some stuff about that, however that is to be expected.

Many of these kind of changes take time, and require multiple interactions.

See Go or .NET tooling bootstraping, all the years that took MaximeVM to evolve into GraalVM, Swift evolution versus Objective-C, Java/Kotlin AOT evolution story on Android, and so on.

If only people that really deeply care get to compile from source to try out the JIT and give feedback, it will have even less people trying it out than those that bother with PyPy.

On the other hand, if people who don't care enough to compile it for themselves try it out, the Python devs can be flooded with complaints and bug reports that effectively come down to it being in alpha.

You get both sides (yes, you might limit some who would otherwise try it out).

I think requiring people to compile to try out such a still-fraught, alpha-level feature isn't too onerous. (And that's only from official sources; third parties can offer compiled versions to their hearts' content!)

PyPy seems soo underrated....

So much work, and so little recognition :-/ I was looking forward to trying it out but moved off python before the libraries I used were compatible (sqlalchemy I believe was the one I was really wanting...)

PyPy still has GIL, though. In my practice (before I jumped ship to Go) PyPy never really made real code much faster. You still had to play games with shared memory to achieve multi-threading, and/or run multiple copies of a server process.

They should have just tried to go GIL-less instead of wasting time on trying transactional memory.

They did. They kicked off the STM project over a decade ago and produced a handful of releases. The project died on the vine:

> STM was a research project that proved that the idea is possible. However, the amount of user effort that is required to make programs run in a parallelizable way is significant, and we never managed to develop tools that would help in doing so. At the moment we're not sure if more work spent on tooling would improve the situation or if the whole idea is really doomed. The approach also ended up adding significant overhead on single threaded programs, so in the end it is very easy to make your programs slower. (We have some money left in the donation pot for STM which we are not using; according to the rules, we could declare the STM attempt failed and channel that money towards the present GIL removal proposal.)

https://pypy.org/posts/2017/08/lets-remove-global-interprete...

> those that bother with PyPy

Which itself needed to be compiled from source the first time I tried it. All the hours of Mandelbrot were worth the spectacular speedup.

There are portable downloads of PyPy here https://pypy.org/download.html

It sounds like their point is that there's precedent for things in early stages only being provided as source, with binary releases coming later when things are more stable

Which was a thing several years ago, not in 2024.

As mentioned I don't have any issues compiling it myself, more of an adoption kind of remark.

Related: It's a mystery to me why Python doesn't provide interpreter binaries for linux. (They do for Windows)

Linux isn’t one target, but a matrix of targets: providing binaries for Linux means picking architectures, libc versions and variants, OpenSSL versions and forks, etc. against which to canonicalize. This also has downstream implications, e.g. a CPython binary with a static build of OpenSSL might contain an OpenSSL vulnerability that CPython is now on the hook for remediating (rather than delegating that remediation responsibility to downstream distributions).

Some of this complexity is also true for Windows, but Linux’s (good!) diversity makes it a bigger challenge.

Valid, but you can make three binaries that will work for the most common use cases. (I've done this) For a project as big as this, doing so is trivial, vice offloading it onto the users.

> offloading it onto the users

Not end users but distro maintainers.

If you run almost any Linux distro imaginable, it has Python already in the distro.

If you want to try a version that's not yet even supported by the unstable branch of your distro, or not available on some PPA, etc, you need to be well enough versed in its dependencies to build it yourself.

Alpha-quality software, of course, benefits from more eyes, but it mostly needs certain kinds of eyes.

I haven't really used it in anger, but I believe `uv` can install pre-compiled python binaries (from... somewhere?). Maybe also pyenv?

With pyenv it's built from source. That's pretty seamless if you have the build dependencies installed. With rye and uv you get the prebuilt portable versions from indygreg. Those have some significant deviations from what you get with a system package or pyenv built version, so there can be some caveats.

Why libc can't converge to a long-term stable version is another mystery.

What do you mean? Glibc is well known for its commitment to compatibility. But that isn’t the issue: the issues are that there’s more than one libc, and that consumers actually want incompatible changes sometimes.

That's not usually the issue. Glibc is used by 90% of main distros, notable exception is alpine used in docker images.

The problem is Linux ecosystem's fixation on "build environment = runtime environment" idea, making it incredibly difficult to build against older versions of glibc (e.g. Ubuntu 22.04) if you're using something new such as latest Arch. This is not a problem on macOS, you can use an SDK for a particular OS version on any other recent version.

Because Linux is a kernel and doesn't have a standardized userland to link against.

Check out IndyGreg's portable Python builds. They're used by Rye and uv.

https://gregoryszorc.com/docs/python-build-standalone/main/

why would they? On Linux yoi typically use thr distribution provided binary or if you really need something newer a container

On Linux you typically clone a project that specifies a specific python version (or range of versions), so you install the right version somewhere and set up your virtualenv to use it. At least in my world.

Using distro linux for non-OS code is IMO a pain. Risk to break your OS, and the distro Py may be old.

They are saving the JIT on release for 3.14! Sssshhhhhh

Pi-thon

not sure if there will distributed in homebrew et al but at least "Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers" [0]

[0] https://docs.python.org/3.13/whatsnew/3.13.html#free-threade...

Yes, that is my point, GIL free is available, but not wit JIT enabled.

Hopefully Macports will decide to offer it.

It will happen in one of the later releases. They might not be ready for widespread testing from people who are not willing to build from source, yet.

Reply on news.ycombinator.com