Everything you need to know about Python 3.13 – JIT and GIL went up the hill | Hacker News Reader

Everything you need to know about Python 3.13 – JIT and GIL went up the hill

drew.silcock.dev

397 pointschmaynard203 comments

Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity?

What is the real world benefit we will get in return?

In the rare case where I need to max out more than one CPU core, I usually implement that by having the OS run multiple instances of my program and put a bit of parallelization logic into the program itself. Like in the mandelbrot example the author gives, I would simply tell each instance of the program which part of the image it will calculate.

There will be consumer chips with 64 cores before long

There is an argument that if you need in process multithreading you should use a different language. But a lot of people need to use python because everything else they’re doing is in python.

There are quite a few common cases where in process multi threading is useful. The main ones are where you have large inputs or large outputs to the work units. In process is nice because you can move the input or output state to the work units instead of having to copy it.

One very common case is almost all gui applications. Where you want to be able to do all work on background threads and just move data back and forth from the coordinating ui thread. JavaScript’s lack of support here, outside of a native language compiled into emscripten, is one reason web apps are so hard to make jankless. The copies of data across web workers or python processes are quite expensive as far as things go.

Once a week or so, I run into a high compute python scenario where the existing forms of multiprocessing fail me. Large shared inputs and or don’t want the multiprocess overhead; but GIL slows everything down.

> Where you want to be able to do all work on background threads and just move data back and forth from the coordinating ui thread. JavaScript’s lack of support here, outside of a native language compiled into emscripten, is one reason web apps are so hard to make jankless

I thought transferring array buffers through web workers didn’t involve any copies of you actually transferred ownership:

    worker.postMessage(view.buffer, [view.buffer]);

I can understand that web workers might be more annoying to orchestrate than native threads and the like but I’m not sure that it lacks the primitives to make it possible. More likely it’s really hard to have a pauseless GC for JS (Python predominantly relies on reference counting and uses gc just to catch cycles).

This is true, but when do you really work with array buffers in Javascript? The default choice for whatever it is that you're doing is almost always something else, save for a few edge cases, and then you're stuck trying to bend your business logic to a different data type.

That’s a choice you get to make and probably depends on your problem domain and other things. For example when I was writing R2 it was all ArrayBuffers up and down the stack. And you could use something like capnproto or flat buffers for managing your object graph within an array buffer. But yeah, being able to transfer custom object graphs would be more powerful.

There is this assumption in these discussions that anything consuming significant CPU must necessarily have a simple interface that’s easy to reduce to a C-level ABI, like calling an ML library on an image, a numpy function on two big matrices or some encryption function. Therefore it is trivial to just move these to native code with an easy, narrow interface.

This assumption is incorrect. There are plenty of problems that consist entirely of business logic manipulating large and complex object graphs. “Just rewrite the hot function in rust, bro” and “just import multiprocessing, bro” are functionally identical to rewriting most of the application for these.

The performance work of the last few years, free threading and JIT are very valuable for these. All the rest is already written in C.

It's a good assumption though, because it keeps (in this case kept) closed the door to the absolutely nightmarish landscape of "multithreading to the masses". Those who made it open probably see it better, but, imo and ime, it should remain closed. Maybe they'll manage to handle it this time, but I'm 95% sure it's gonna be yet another round of ass pain for the world of python.

> “Just rewrite the hot function in rust, bro” and “just import multiprocessing, bro” are functionally identical to rewriting most of the application for these.

Isn't "just use threads, bro" likely to be equally difficult?

no. it's much harder

otoh, if all of your time is spent in python code and you have performance issues, it's time to rewrite in a different language. Correct multi threaded coffee is take hard and python is inherently really slow. the inherent complexity of multi threaded code is enough that you should just write the single threaded version in a language that's 10x faster (of which there are many)

Is this some internal cloudflare feature flag or can everybody pass ArrayBuffers zero-copy via service bindings?

(random question, totally understand if you're not the right person to ask)

It's not a cloudflare thing: https://developer.mozilla.org/en-US/docs/Web/API/Worker/post...

My hunch is that in just a few years time single core computers will be almost extinct. Removing the GIL now feels to me like good strategic preparation for the near future.

Single core computers yes. Single core containers though..

Single core containers are also a terrible idea. Life got much less deadlocked as soon as there were 2+ processors everywhere.

(Huh, people like hard OS design problems for marginal behavior? OSes had trouble adopting SMP but we also got to jettison a lot of deadlock discussions as soon as there was CPU 2. It only takes a few people not prioritizing 1 CPU testing at any layer to make your 1 CPU container much worse than a 2 VCPU container limited to a 1 CPU average.)

It's actually quite difficult to get a "single core" container (ie: a container with access to only one logical processor).

When you set "request: 1" in Kubernetes or another container manager, you're saying "give me 1 CPU worth of CPU time" but if the underlying Linux host has 16 logical cores your container will still see them.

Your container is free to use 1/16th of each of them, 100% of one of them, or anything in-between.

You might think this doesn't matter in the end but it can if you have a lot of workloads on that node and those cores are busy. Your single threaded throughout can become quite compromised as a result.

> if you have a lot of workloads on that node and those cores are busy. Your single threaded throughout can become quite compromised as a result.

While yes this can cause a slowdown, wouldn't it still happen if each container thought it had a single core?

Only if you have more containers than cores.

That depends on what your scheduler does. Having one virtual core doesn't necessarily mean you always get the same physical core.

Also you said "a lot of workloads" so yes probably more containers than cores.

Most of my pods have a CPU request >= 1 so more containers than cores is rare. But obviously that really depends on your workload(s).

I don't think the scheduler picking a different core matters much unless your workload is super cache sensitive. My point is more about access to single threaded performance. If you have a single threaded workload (ex: an ffmpeg audio encode) and you want it to be able to access as many cycles from a single core as possible, it isn't always as simple as request: 1

It's easy, though?

On Docker, --cpuset-cpus=0 will pin the container to the first core.

K8s: https://kubernetes.io/docs/tasks/administer-cluster/cpu-mana...

CPU affinity and pinning is something I think you should be able to achieve without too much hassle.

I think the point was this isn’t the norm though. If you know you need to be pinned to a core you CAN configure kubernetes to do so but it’s not the default and therefore you are unknowingly leaving performance on the floor

I'm quite certain you'd leave more performance on the table by pinning in general and on average.

Just let the CPU scheduler do its job. Unless you know better, in which case, by all means go ahead and allocate computational resources manually. I don't see a way to make that a sensible default, though.

> It's easy though

Neat, I didn't know it was a single flag in Docker.

The k8s method you linked definitely has some caveats as it doesn't allow this scheduling at the pod level, and requires quite a bit of fiddling to get working (atleast on GKE). This isn't even available if you use a fully managed setup like autopilot.

Maybe my expectations just aren't realistic but "easy" to me would mean I put the affinity right next to my CPU request in the podSpec :/

It depends what you mean by extinct.

I can't think of any actual computer outside of embedded that has been single core for at least a decade. The Core Duo and Athlon X2 were released almost 20 years ago now and within a few years basically everything was multicore.

(When did we get old?)

If you mean that single core workloads will be extinct, well, that's a harder sell.

Yeah, I just checked and even a RaspberryPi has four cores these days. So I guess they went extinct a long time ago!

Even many microcontrollers have multiple cores nowadays. It’s not the norm just yet, though.

Yes, but:

* Most of the programs I write are not (trivially) parallelizable, and a the bottleneck is still a single core performance

* There is more than one process at any time, especially on servers. Other cores are also busy and have their own work to do.

Yes, but:

1. Other people with different needs exist.

2. That's why we have schedulers.

> My hunch is that in just a few years time single core computers will be almost extinct.

Single core computers are already functionally extinct, but single-threaded programs are not.

Depends on the OS, on Windows or Android, even single processes have multiple threads under the hood.

What you’re describing is basically using MPI in some way, shape or form. This works, but also can introduce a lot of complexity. If your program doesn’t need to communicate, then it’s easy. But that’s not the case for all programs. Especially once we’re talking about simulations and other applications running on HPC systems.

Sometimes it’s also easier to split work using multiple threads. Other programming languages let you do that and actually use multiple threads efficiently. In Python, the benefit was just too limited due to the GIL.

> Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity?

This was the original reason for CPython to retain GIL for very long time, and probably true for most of that time. That's why the eventual GIL removal had to be paired with other important performance improvements like JIT, which was only implemented after some feasible paths were found and explicitly funded by a big sponsor.

That is the official story. None of it has materialized so far.

Python development is done in public so you can just benchmark against the development version to see its improvement. In fact, daily benchmarks are already posted to [1]; it indicates around 20% improvement (corresponding to 1.25x in the table) since 3.10. The only thing you can't easily verify is that whether GIL was indeed historically necessary in the past.

[1] https://github.com/faster-cpython/benchmarking-public

> What is the real world benefit we will get in return?

If you have many CPU cores and an embarrassingly parallel algorithm, multi-threaded Python can now approach the performance of a single-threaded compiled language.

The question really is if one couldn't make multiprocess better instead of multithreaded. I did a ton of MPI work with python ten years ago already.

What's more I am now seeing in Julia that multithreading doesn't scale to larger core counts (like 128) due to the garbage collector. I had to revert to multithreaded again.

I assume you meant you had to revert to multiprocess?

Yes exactly. Thanks.

You could already easily parallelize with the multiprocessing module.

The real difference is the lower communication overhead between threads vs. processes thanks to a shared address space.

Easily is an overstatement. Multiprocessing is fraught with quirks.

Well I once had an analytics/statistics tool that regularly chewed through a couple GBs of CSV files. After enough features had been added it took almost 5 minutes per run which got really annoying.

It took me less than an hour to add multiprocessing to analyze each file in its own process and merge the results together at the end. The runtime dropped to a couple seconds on my 24 thread machine.

It really was much easier than expected. Rewriting it in C++ would have probably taken a week.

In F#, it would just be

    let results = files |> Array.Parralel.map processFile

Literally that easy.

Earlier this week, I used a ProcessPoolExecutor to run some things in their own process. I needed a bare minimum of synchronization, so I needed a queue. Well, multiprocessing has its own queue. But that queue is not joinable. So I chose the multiprocessing JoinableQueue. Well, it turns out that that queue can't be used across processes. For that, you need to get a queue from the launching process' manager. That Queue is the regular Python queue.

It is a gigantic mess. And yes, asyncio also has its own queue class. So in Python, you literally have a half a dozen or so queue classes that are all incompatible, have different interfaces, and have different limitations that are rarely documented.

That's just one highlight of the mess between threading, asyncio, and multiprocessing.

Well I'm not here to debate the API cleanliness, I just wanted to point out to OP that Python can utilize multicore processors without threads ;)

Here is the part of multiprocessing I used:

  with Pool() as p:
      results = p.map(calc_func, file_paths)

So, pretty easy too IMO.

Fraught with quirks sounds quite ominous. Quuuiiirkkksss.

I agree though.

That's not really correct. Python is by far the slowest mainstream language. It is embarrassingly slow. Further more, several mainstream compiled languages are already multicore compatible and have been for decades. So comparing against a single-threaded language or program doesn't make sense.

All this really means is that Python catches up on decades old language design.

However, it simply adds yet another design input. Python's threading, multiprocessing, and asyncio paradigms were all developed to get around the limitations of Python's performance issues and the lack of support for multicore. So my question is, how does this change affect the decision tree for selecting which paradigm(s) to use?

> Python's threading, multiprocessing, and asyncio paradigms were all developed to get around the limitations of Python's performance issues and the lack of support for multicore.

Threading is literally just Python's multithreading support, using standard OS threads, and async exists for the same reason it exists in a bunch of languages without even a GIL: OS threads have overhead, multiplexing IO-bound work over OS threads is useful.

Only multiprocessing can be construed as having been developed to get around the GIL.

No, asyncio's implementation exists because threading in Python has huge overhead for switching between threads and because threads don't use more than one core. So asyncio was introduced as a single threaded solution specifically for only network-based IO.

In any other language, async is implemented on top of the threading model, both because the threading model is more efficient than Python's and because it actually supports multiple cores.

Multiprocessing isn't needed in other languages because, again, their threading models support multiple cores.

So the three, relatively incompatible paradigms of asyncio, threading, and multiprocessing specifically in Python are indeed separate attempts to account for Python's poor design. Other languages do not have this embedded complexity.

> In any other language, async is implemented on top of the threading model

There are a lot of other languages. Javascript for example is a pretty popular language where async on a single threaded event loop has been the model since the beginning.

Async is useful even if you don't have an interpreter that introduces contention on a single "global interpreter lock." Just look at all the languages without this constraint that still work to implement async more naturally than just using callbacks.

Threads in Python are very useful even without removing the gil (performance critical sections have been written as extension modules for a long time, and often release the gil).

> are indeed separate attempts to account for Python's poor design

They all have tradeoffs. There are warts, but as designed it fits a particular use case very well.

Calling Python's design "poor" is hubris.

> So my question is, how does this change affect the decision tree for selecting which paradigm(s) to use?

The only effect I can see is that it reduces the chances that you'll reach for multiprocessing, unless you're using it with a process pool spread across multiple machines (so they can't share address space anyway)

> Calling Python's design "poor" is hubris.

Not in the least. Python is a poorly designed language by many accounts. Despite being the most popular language in the world, what language has it significantly influenced? None of note.

Who cares about how many other languages a language has influenced? If that was a metric of any consideration we all would write Algol or something. Programming languages are tools, tools to help you perform a task.

> Python is a poorly designed language by many accounts

Hubris isn't rare.

> what language has it significantly influenced?

I can think of at least 1 language designer[1] who doesn't think it's "poorly designed," based on it's significant impact on what they're currently working on[2]

1. https://en.m.wikipedia.org/wiki/Chris_Lattner 2. https://www.modular.com/mojo

>Python is by far the slowest mainstream language. It is embarrassingly slow.

Oh? It is by far the fastest language for me. No languages comes close on the time from starting to write, to have code that runs. For me that time far outweighs the execution time, so it is a lot more important.

may i ask in what field do you specialize? Because any modern language i can think of, is one "project init" command away from "nothing" to "running"

The biggest use case (that I am aware of) of GIL-less Python is for parallel feeding data into ML model training.

* PyTorch currently uses `multiprocessing` for that, but it is fraught with bugs and with less than ideal performance, which is sorely needed for ML training (it can starve the GPU).

* Tensorflow just discards Python for data loading. Its data loaders are actually in C++ so it has no performance problems. But it is so inflexible that it is always painful for me to load data in TF.

Given how hot ML is, and how Python is currently the major language for ML, it makes sense for them to optimize for this.

> What is the real world benefit we will get in return?

None. I've been using Python "in anger" for twenty years and the GIL has been a problem zero times. It seems to me that removing the GIL will only make for more difficulty in debugging.

As always, it depends a lot on what you're doing, and a lot of people are using Python for AI.

One of the drawbacks of multi-processing versus multi-threading is that you cannot share memory (easily, cheaply) between processes. During model training, and even during inference, this becomes a problem.

For example, imagine a high volume, low latency, synchronous computer vision inference service. If you're handling each request in a different process, then you're going to have to jump through a bunch of hoops to make this performant. For example, you'll need to use shared memory to move data around, because images are large, and sockets are slow. Another issue is that each process will need a different copy of the model in GPU memory, which is a problem in a world where GPU memory is at a premium. You could of course have a single process for the GPU processing part of your model, and then automatically batch inputs into this process, etc. etc. (and people do) but all this is just to work around the lack of proper threading support in Python.

By the way, if anyone is struggling with these challenges today, I recommend taking a peek at nvidia's Triton inference server (https://github.com/triton-inference-server/server), which handles a lot of these details for you. It supports things like zero-copy sharing of tensors between parts of your model running in different processes/threads and does auto-batching between requests as well. Especially auto-batching gave us big throughput increase with a minor latency penalty!

Machine learning people not call their thing Triton challenge (IMPOSSIBLE)

This (Nvidia’s) triton predates openAI’s by a few years.

> For example, imagine a high volume, low latency, synchronous computer vision inference service.

I'm not in this space and this is probably too simplistic, but I would think pairing asyncio to do all IO (reading / decoding requests and preparing them for inference) coupled with asyncio.to_thread'd calls to do_inference_in_C_with_the_GIL_released(my_prepared_request), would get you nearly all of the performance benefit using current Python.

> Removing the GIL sounds like it will make typical Python programs slower and will introduce a lot of complexity?

There is a lot of Python code that either explicitly (or implicitly) relies on the GIL for correctness in multithreaded programs.

I myself have even written such code, explicitly relying on the GIL as synchronization primitive.

Removing the GIL will break that code in subtle and difficult to track down ways.

The good news is that a large percentage of this code will stay running on older versions of python (2.7 even) and so will always have a GIL around.

Some of it however will end up running on no-GIL python and I don't envy the developers who will be tasked tracking down the bugs - but probably they will run on modern versions of python using --with-gil or whatever other flag is provided to enable the GIL.

The benefit to the rest of the world then is that future programs will be able to take advantage of multiple cores with shared memory, without needing to jump through the hoops of multi-process Python.

Python has been feeling the pain of the GIL in this area for many years already, and removing the GIL will make Python more viable for a whole host of applications.

I really find a bummer that there is such a resistance to make JIT enable versions available for download, alongside the other variants.

Naturally I can easily compile my own Python 3.13 version, no biggie.

However from my experience, this makes many people that could potentially try it out and give feedback, don't care and rather wait.

not sure if there will distributed in homebrew et al but at least "Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers" [0]

[0] https://docs.python.org/3.13/whatsnew/3.13.html#free-threade...

Hopefully Macports will decide to offer it.

Yes, that is my point, GIL free is available, but not wit JIT enabled.

JIT is in weird place now because according to a recent talk from PyCON US by it’s author, the tier 2 optimizer that prepares code for JIT reduces performance by 20% and JIT just recovers that performance loss.

A lot of langugage is still not optimized by tier 2 [1] and even less has copy and patch templates for JIT. And JIT itself currently has some memory management issues to iron out.

[1]: https://github.com/python/cpython/issues/118093

Talk by Brandt Butcher was there but it was made private:

https://www.youtube.com/watch?v=wr0fVU3Ajwc

I have seen some stuff about that, however that is to be expected.

Many of these kind of changes take time, and require multiple interactions.

See Go or .NET tooling bootstraping, all the years that took MaximeVM to evolve into GraalVM, Swift evolution versus Objective-C, Java/Kotlin AOT evolution story on Android, and so on.

If only people that really deeply care get to compile from source to try out the JIT and give feedback, it will have even less people trying it out than those that bother with PyPy.

> those that bother with PyPy

Which itself needed to be compiled from source the first time I tried it. All the hours of Mandelbrot were worth the spectacular speedup.

Which was a thing several years ago, not in 2024.

As mentioned I don't have any issues compiling it myself, more of an adoption kind of remark.

There are portable downloads of PyPy here https://pypy.org/download.html

It sounds like their point is that there's precedent for things in early stages only being provided as source, with binary releases coming later when things are more stable

On the other hand, if people who don't care enough to compile it for themselves try it out, the Python devs can be flooded with complaints and bug reports that effectively come down to it being in alpha.

You get both sides (yes, you might limit some who would otherwise try it out).

I think requiring people to compile to try out such a still-fraught, alpha-level feature isn't too onerous. (And that's only from official sources; third parties can offer compiled versions to their hearts' content!)

PyPy seems soo underrated....

So much work, and so little recognition :-/ I was looking forward to trying it out but moved off python before the libraries I used were compatible (sqlalchemy I believe was the one I was really wanting...)

PyPy still has GIL, though. In my practice (before I jumped ship to Go) PyPy never really made real code much faster. You still had to play games with shared memory to achieve multi-threading, and/or run multiple copies of a server process.

They should have just tried to go GIL-less instead of wasting time on trying transactional memory.

They did. They kicked off the STM project over a decade ago and produced a handful of releases. The project died on the vine:

> STM was a research project that proved that the idea is possible. However, the amount of user effort that is required to make programs run in a parallelizable way is significant, and we never managed to develop tools that would help in doing so. At the moment we're not sure if more work spent on tooling would improve the situation or if the whole idea is really doomed. The approach also ended up adding significant overhead on single threaded programs, so in the end it is very easy to make your programs slower. (We have some money left in the donation pot for STM which we are not using; according to the rules, we could declare the STM attempt failed and channel that money towards the present GIL removal proposal.)

https://pypy.org/posts/2017/08/lets-remove-global-interprete...

It will happen in one of the later releases. They might not be ready for widespread testing from people who are not willing to build from source, yet.

Related: It's a mystery to me why Python doesn't provide interpreter binaries for linux. (They do for Windows)

Linux isn’t one target, but a matrix of targets: providing binaries for Linux means picking architectures, libc versions and variants, OpenSSL versions and forks, etc. against which to canonicalize. This also has downstream implications, e.g. a CPython binary with a static build of OpenSSL might contain an OpenSSL vulnerability that CPython is now on the hook for remediating (rather than delegating that remediation responsibility to downstream distributions).

Some of this complexity is also true for Windows, but Linux’s (good!) diversity makes it a bigger challenge.

Valid, but you can make three binaries that will work for the most common use cases. (I've done this) For a project as big as this, doing so is trivial, vice offloading it onto the users.

> offloading it onto the users

Not end users but distro maintainers.

If you run almost any Linux distro imaginable, it has Python already in the distro.

If you want to try a version that's not yet even supported by the unstable branch of your distro, or not available on some PPA, etc, you need to be well enough versed in its dependencies to build it yourself.

Alpha-quality software, of course, benefits from more eyes, but it mostly needs certain kinds of eyes.

I haven't really used it in anger, but I believe `uv` can install pre-compiled python binaries (from... somewhere?). Maybe also pyenv?

With pyenv it's built from source. That's pretty seamless if you have the build dependencies installed. With rye and uv you get the prebuilt portable versions from indygreg. Those have some significant deviations from what you get with a system package or pyenv built version, so there can be some caveats.

Why libc can't converge to a long-term stable version is another mystery.

What do you mean? Glibc is well known for its commitment to compatibility. But that isn’t the issue: the issues are that there’s more than one libc, and that consumers actually want incompatible changes sometimes.

That's not usually the issue. Glibc is used by 90% of main distros, notable exception is alpine used in docker images.

The problem is Linux ecosystem's fixation on "build environment = runtime environment" idea, making it incredibly difficult to build against older versions of glibc (e.g. Ubuntu 22.04) if you're using something new such as latest Arch. This is not a problem on macOS, you can use an SDK for a particular OS version on any other recent version.

why would they? On Linux yoi typically use thr distribution provided binary or if you really need something newer a container

Using distro linux for non-OS code is IMO a pain. Risk to break your OS, and the distro Py may be old.

On Linux you typically clone a project that specifies a specific python version (or range of versions), so you install the right version somewhere and set up your virtualenv to use it. At least in my world.

Because Linux is a kernel and doesn't have a standardized userland to link against.

Check out IndyGreg's portable Python builds. They're used by Rye and uv.

https://gregoryszorc.com/docs/python-build-standalone/main/

They are saving the JIT on release for 3.14! Sssshhhhhh

Pi-thon

The performance degradation with nogil is quoted as 20%. It can easily be as much as 50%.

The JIT does not seem to help much. All in all a very disappointing release that may be a reflection of the social and corporate issues in CPython.

A couple of people have discovered that they can milk CPython by promising features, silencing those who are not 100% enthusiastic and then underdeliver. Marketing takes care of the rest.

Could you clarify what/who you mean by the final sentence. It gives the impression you didn't take on board the articles mention about the three phases.

Why are you disappointed? Do you think that the progress should be faster? Or that this work should never have been started? Or that they should wait until it works better before integrering in master and having it in a mainline release?

The last one.

I think they explained why they are disappointed

I remember first discussions about removing the GIL back in 2021 and a lot of initial confusion about what the implications would be. This is a great summary if, like me, you weren’t satisfied with the initial explanations given at the time.

2021 wasn't the first discussion about that.

You can find forum and Reddit posts going back 15-20 years of people attempting to remove the GIL, Guido van Rossum just made the requirement that single core performance cannot be hurt by removing it, this made ever previous attempt fail in the end

I.e. https://www.artima.com/weblogs/viewpost.jsp?thread=214235

Did this attempt manage to preserve single-threaded performance, or was the requirement dropped?

He folded.

The patches dropped some unrelated dead weight such that the effect is not as bad.

The effect is still 20-50% slowdown for single thread, even with the purported unrelated speedups to make the feature more palatable.

That is absolutely in the range of previous attempts, which were rejected! The difference here is that its goes in now to gratify Facebook.

Facebook or Microsoft?

This sounds silly but I’ve actually turned off garbage collection in short running, small memory programs and gotten a big speed boost.

I wonder if that’s something they could automate? I’m sure there are some weird risks with that. Maybe a small program ends up eating all your memory in some edge case?

You'd run into the halting problem. Maybe for some small subset of programs it'd be possible to prove a short runtime, but in general it wouldn't be, so this type of automation wouldn't be impossible.

It sounds like maybe you want GCs to be very tunable? That way, developers and operators can change how it runs for a given workload. That's actually one of the (few) reasons I like Java, its ability to monitor and tune its GC is awesome.

No one GC is going to be optimal for all workloads/usages. But it seems like the prevailing thought is to change the code to suit the GC where absolutely necessary, instead of tuning the GC to the workload. I'm not sure why that is?

Reliable implementation most likely involves solving the halting problem.

Disagree, there are practical trivial subsets of the halting problem to which I imagine many short-running scripts would conform.

Most short running scripts call into the standard library or into other packages. I think the trivial subset is too small to bother with such an optimisation.

Turn on GC after the first sbrk

The first alloc will be really early on in even trivial Python programs, so that’s not a good heuristic. It’s hard to pick a good heuristic in this area because the line between startup time and runtime is so blurry in platforms that support runtime/deferred import and eval.

So is it impossible to optimize the no-GIL case more really? 20% slower sounds like a lot really.

Where it says that? Its simply that Python releases features in yearly cycles and thats what was completed for release.

Idea is to let people experiment with no-GIL to see what it breaks while maintainers and outside contractors improve the performance in future versions.

No that was not the idea. The feature went in under the assumption that the single thread slowdown would be offset by other minor speed improvements.

That was literally the official reason why it was accepted. Now we have slowdowns ranging from 20-50% compared to Python 3.9.

What outside contractors would fix the issue? The Python ruling class has chased away most people who actually have a clue about the Python C-API, which are now replaced by people pretending to understand the C-API and generating billable hours.

Just a minor correction: it's looking like the release will be 7th October, back from 2nd October, for the reasons discussed here:

https://discuss.python.org/t/incremental-gc-and-pushing-back...

Thanks for pointing this out, I hadn't seen that – I've just pushed an update to reflect this.

Can someone explain this part:

> What happens if multiple threads try to access / edit the same object at the same time? Imagine one thread is trying to add to a dict while another is trying to read from it. There are two options here

Why not just ignore this fact, like C and C++? Worst case this is a datarace, best case the programmer either puts the lock or writes a thread safe dict themselves? What am I missing here?

Memory safety, heap integrity and GC correctness i guess. If you ignore facts like C your language will be as safe as C, except it's worse because at least C doesn't silently rearrange your heap in background.

There is an expectation of not dealing with data races from Python code. Apart from being a language where people expect these things to not be an issue it is also the behaviour with thw Gil in place and would be breaking

Let me preface this by saying I have no source to prove what I’m about to say, but Guido van Rossum aimed to create a programming language that feels more like a natural language without being just a toy language. He envisioned a real programming language that could be used by non-programmers, and for this to work it couldn’t contain the usual footguns.

One could argue that he succeeded, considering how many members of the scientific community, who don’t primarily see themselves as programmers, use Python.

All the answers were good, but I think this explained it the best. Thank you

But the performance motivations for removing the GIL seem at odds with this. I feel like the subset of Python users who care about the GIL and the subset who are "non-programmers" are entirely disjoint.

I guess I am one of these people and have zero idea on how to write code in parallel programming. Would GIL removal benefit me, given that it seems like it would hurt performance?

Why not just use C and C++ at that point? People use Python because they don't want to manage data races or re-implement a hash table.

This was not a gotcha, sorry if it came across that way. It was a genuine question

We don't want Python programs to become the same kind of security swiss cheese as C/C++ programs.

It's harder to ignore the problem in Python, because reference counting turns every read into a write, incrementing and then decrementing the recount of the object you read. For example, calling "my_mutex.lock()" has already messed with the recount on my_mutex before any locking happens. If races between threads could corrupt those refcounts, there's no way you could code around that. Right now the GIL protects those refcounts, so without a GIL you need a big change to make them all atomic.

Oh wow, I haven't thought about refcounts on mutexes

> Worst case this is a datarace [... w]hat am I missing here?

That the worst case being memory unsafety and a compromised VM is not acceptable? Especially for a language as open to low-skill developers as Python?

Hm I get the memory safety part, but could you elaborate on the compromised VM part? I'm not sure I understand that.. Specifically I don't understand what you mean by VM here

Once you compromise memory safety you have access to the entire Python runtime, there is no separation.

I am scientific Python programmer. 99% to 100% of my programs require parallelism, but it is ALWAYS embarrassingly trivial parallelism, nothing is ever mutated and I never need locks. Right now I am forced to use multiprocessing to get the performance, with all problems of multiprocessing, the major one being that I need to use more memory. For me using multithreading could mean the difference between running out of memory and not running out memory. The GIL removal matters for people like me, the proponents of the GIL removal comes from the scientific community.

Slightly off topic: does anyone have a link to recent work done toward automatic parallelization?

(Write single threaded code and have a compiler create multithreaded code)

https://en.m.wikipedia.org/wiki/Automatic_parallelization_to...

Rust's Rayon could probably have a mode with additional heuristics, so it wouldn't multi-thread if it guessed the memory usage increase or overhead would be too much.

Not really automatic, but for some iterator chains you can just slap an adapter on.

Currently you have to think and benchmark, but for some scripting type applications the increased overhead of the heuristics might be justified, as long as it was a new mode.

Reposting this answer as it’s relevant here too - we already do this - a modern superscalar CPU is essentially an engine for turning sequential code into 6-8 (or more!) simultaneous operations that are dispatched all at once. Downstream, some 4-8 (or more!) ops are retired at once, their operations complete. This is all in a single core.

Oh it's like the rhyme "Jack and Jill went up the hill".

I spent all day not knowing whether "up the hill" meant they shipped or didn't ship. So they shipped, right? Or they shipped a JIT but removed the GIL?

I wonder if they could now add a way for the interpreter to automatically find instances where it could run your code in parallel?

You’d think certain patterns could be probably safe and the interpreter could take the initiative.

Is there a term for this concept?

No way that's possible or even desirable with Python's OOP and mutable nature and scoping structure.

Automatic parallelization, as someone else linked to.

It is another one of those things that many programmers ask "Why hasn't this been tried?" and the answer is, it has, many times over. It just failed so hard and so fast you've never heard of the results. Toy problems speed up by less than you'd hope and real code gets basically nothing. Your intuition says your code is full of parallelization opportunities; your intuition turns out to be wrong. A subset of the general problem that even very experienced developers still never get great at knowing how code will perform without simply runninga profiler.

It has failed hard and fast in languages much friendlier to the process than Python. Proving something is truly parallel-safe in Fortran is hard enough. Proving it in Python is effectively impossible, you just don't know when something is going to dynamically dynamic the dynamics on you.

> you just don't know when something is going to dynamically dynamic the dynamics on you.

I'm going to use that!

I’ll take a different tact than previous answers and say we already do this - a modern superscalar CPU is essentially an engine for turning sequential code into 6-8 (or more!) simultaneous operations that are dispatched all at once. Downstream, some 4-8 (or more!) ops are retired at once, their operations complete. This is all in a single core.

Is it related to the mojo language work? would that affect how they track the language features?

What an incredible title.

Experimentally JIT WHILE there is totally stable , 4-100x faster, almost 100% compatible PyPy exist and all they need is adopt it but didn't due to some politics.

Yeah right..

I don't follow Python closely. So why PyPy isn't adopted?

The biggest reasons is that it isn't compatible with the python C API. So any library that uses the C API would have to be rewritten.

They have gotten much better about that recently and it's much much more compatible now.

The main reason GVR said is "May be this is future of Python , but not now" . At that time pypy was already stable at 2.7.x version but python 2-3 split happen. PyPy team have to rewrite everything for Python3 and took them a while to catchup. I had managed to talk with one ex pypy developer that the main reason GVR and Python community do not promote much about PyPy is due to NIH - Not invented here - Since PyPy is developed by seperate but full time computer scientist and researchers, PHDs , not by the Guido community . https://pypy.org/people.html

One problem about popularity of pypy is they don't do any advertisement , promotion which I had critically voiced about it in their community - they moved to Github finally.

Only other problem is CPython Ext , which is compatible but a little bit slower than CPython - that only pain point we have - which could be solved if there are more contributors and users using pypy. Actually Python , written in Python should be the main

Thankfully everyone is free to use PyPy, if they consider it the better solution!

Splitting efforts aren't good . PyPy is decade effort of making JIT Python and they had big success. But it doesn't get the praise by the python community - and they are only doing NIH solutions again and again. (First attemped by Google and they failed - unladen python , now this experiment results dosen't sound good).

Why ignore millions of dollars spent in a decade effort of fulltime PHD researchers' work and doing their own thing?

Yeah NIH is helluva drug.

As others have mentioned - Python exposes very low-level details to C, making plenty of optimizations invalid as programs may make use of it.

Python allows a lot of paradigms that are really difficult to JIT. I have personally seen many instances where PyPy is actually slower than CPython for a basic web app because of this.

It is only slow when you using C-Extension. Basic webapp don't use C-Extensions. If you are going to claim such please provide evidence. We have a chat server in production written in PyPy and Tornadoweb which had sped up 20x from CPython version and test benchmarks against GoLang based Websocket with Channels and Nodejs - The pypy implementation is 30% faster than node and 10-20% faster than Golang . Big plus is PyPy version drops memory usage by 80% . CPython version have around 200MB Ram usage and pypy version only about 40MB.

On Heavy load (10k concurrent test) PyPy and Golang version are stable but Node version stop responding sometimes and packet losses occurs.

You do know that the Tornado implementation of WebSocket uses a c extension?

And if you use FastAPI then you're using Rust for serialisation and validation.

Well implemented c extensions are and cffi ones are fine for PyPy. Those that have problems are ones that depends on private c-api calls Tornado had moved to python asyncio for ages.

Congratulations. You sound like you have a very simple application. It is not only slow with C extensions. You would know this if you had more experience with PyPy on a real codebase.

Then define your real codebase. There is no point continuing this with just baseless talks. There are many projects, livbs in pure python without Any c extensions. PyPy work faster with cffi based libs than python c-extension Based ones too.

It’s worth mentioning that there is a bit more overhead in using multiple processes as opposed to multiple threads, in addition to it being more difficult to share data.

There’s probably a whole generation of programmers (if not two) who don’t know the feeling of shooting yourself in the foot with multithreading. You spend a month on a prototype, then some more to hack it all together for semi-real world situations, polish the edges, etc. And then it falls flat day 1 due to unexpected races. Not a bad thing on itself, transferrable experience is always valuable. And don’t worry, this one is. Enough ecos where it’s not “difficult to share data”.

This. Multi-threading is very prone to nasty, hard to reproduce, bugs if used liberally in a codebase. It really should be used with care, and compartmentalized to areas where it demonstratively brings critical performance improvements.

Let’s not tout abysmal state of multi-tasking in Python as an advantage.

Somehow, it’s perfectly fine in C#, Java, now Go, Rust and many other languages, with relatively low frequency of defects.

That is not all what I did. But there are several languages/ecosystems which are very prone to such issues - notably C and C++. It is critical important (in my opinion) that Python does not regress to anywhere near that state. This might seem like an invalid concern - after all Python is a high-level, memory-safe language - right? The problem is that use of extensions (mostly in C or C++) are very common, and they rely on particular semantics - which are now changing. A vast amount of Python programs make use of such extensions, probably the majority of programs (even if excluding the standard library). Some of the biggest challenges around nogil and multi-threading in Python are mostly around extension/C related stuff. It was notably also on of the main challenges faced by PyPy. So maybe it's actually a little bit tricky to get right - and not just the developers being stupid or lazy ;) I mean in addition to the the usual general trickiness of major changes making to the core of an interpreter relied by thousands of companies/projects, in a wide range of fields, built over decades, with varying level of quality and maintenance....

Right, I should have phrased it better - did not intend to make it sound like criticism of your reply. Was just aiming at the tendency to dismiss valid concerns with "it's actually a good thing we don't have it" or "it can't be done well".

Of course changing the concurrency guarantees the code relies on and makes assumptions about is one of the most breaking changes that can be made to a language, with very unpleasant failure modes.

Understood. There has been some amount of that in the past. And probably this kind of pushback will rise up again as the work starts to materialize. I think some are a bit fearful of the potential bad consequences - and it remains unclear which will materialize and which will be non-issues. And of course some have other things they wish to see improved instead, cause they are satisfied with current state. Actually many will be quite happy with (or at least have accepted) the current state - cause those that were/are not probably do not use Python much!

What I see from the development team and close community so far has been quite trust building for me. Slow and steady, gradual integration and testing with feature flags, improving related areas in preparation (like better/simplified C APIs), etc.

So is Turing completeness. But we can’t just put our heads into the sand and ignore these, as we require both Turing completeness and multi-threading in most cases. This is part of our jobs, and is what makes the whole field an art - both of these properties make reasoning about arbitrary programs harder than what is possible with math.

These two are not equivalent in practice. We’ve created a good set of tools that help one to stay in parts of TC that we agreed to consider relatively sane. TC is much bigger than we actually use. It is an enormous landscape, and we only dare to walk very specific paths on it, even our hardware specializes on methods we tend to use.

A similar set of tools and guardrails has yet to be seen for MT, at least in python-ic category. I think we can agree that relative success of MT in some X doesn’t automagically translate everywhere, because it depends on first principles which are different.

The number one thing I wish was addressed in future version of Python is the semantically significant whitespace.

Python is absolutely the worst language to work in with respect to code formatters. In any other language I can write my code, pressing enter or skipping enter however I want, and then the auto formatter just fixes it and makes it look like normal code. But in python, a forgotten space or an extra space, and it just gives up.

It wouldn't even take much, just add a "end" keyword and the LSP's could just take care of the rest.

GIL and JIT are nice, but please give me end.

I don't have this problem, and I've been writing Python for more than twenty years. Sure, I may have the occasional wrong space somewhere, but it's maybe a few times a month, whereas I'd otherwise have to type "end" for every single block.

I dont think this is a problem anymore in todays world of LSPs and auto formatters. I almost never have to type "end" in Elixir for instance, it is always autocompleted for me.

How does it know when to end a block?

The autoformatter does it based on indentation, but when writing code the editor just inserts it when you open a block (after your cursor), same way } is added when you type { in other languages.

Ahh OK, I see, thanks. I don't really see much difference with whitespace sensitive languages, I've never really had problems with it. Maybe I've been lucky.

The day your wish is fullfilled is the day I stop working with Python. I can't stand all those useless braces everywhere, why are they there at all since good practice mandates proper indentation anyway?

I am at the point where I prefer single quotes for strings, instead of double quotes, just because they feel cleaner. And unfortunately pep8 sometimes mandates double quotes for reasons unknown.

No need for braces. Just add "end" for marking block ending to match the already block starting keyword ":".

A while ago, when thinking about syntax design for a new language, I considered this combination (`:` and `end`, as opposed to `do` and `end` as used by Lua etc).

Are there any languages that use it, or is Python unique in using `:` to begin a block?

Nim uses it too.

IIRC, Python's predecessor (ABC) didn't have the trailing colon but they did some experiments and found it increased readability.

Single quotes are also easier to type on the default layout, no Shift

This suggestion gives me chills. I literally never face this issue. Are you using vim? Theres autoindent and smartindent features you can enable to help you.

Neovim + ruff lsp. I have gone through many formatters to try and get this better, but it is always worse than any other language where whitespace is not semantic.

The one nice perk about the state of things atm in python is I can very easily filter out devs by their choice of python IDE (or lack thereof).

    >>> from __future__ import braces
    SyntaxError: not a chance

Thank you, but I rather not inject a tool that hasn't been updated in 6 years into my build chain. Thats how we do things in the Javascript world and frankly it sucks.

This is a joke that's actually built into Python. The __future__ module is where you can import/enable features that will become the default in future versions. The point it makes by giving "SyntaxError: Not a chance" is that Python will never add braces.

And IMO for good reason. It makes the code so much cleaner and it's not like it particularly takes effort to indent your code correctly, especially since any moderately competent editor will basically do it for you. Tbh I actually find it much less effort than typing braces.

Whitespace is semantically significant in nearly all modern programming languages, the difference is that with Python it is completely significant for both the humans and the tools - it is syntactically significant.

I've actively used Python for a quarter of a century (solo, with small teams, with large teams, and with whole dev orgs) and the number of times that not having redundant block delimiters has caused problems is vanishingly small and, interestingly, is on par with the number of times I've had problems with redundant block delimiters getting out of sync, i.e. some variation of

  if (a > b)
    i++;
    j++;

Anytime I switch from Python to another language for awhile, one of the first annoying things is the need to add braces everywhere, and it really rubs because you are reminded how unnecessary they are.

Anyway, you can always write #end if you'd like. ;-)

The parent’s point was that you don’t have to care about white space when composing code in other languages since the LSP can auto format. So you could theoretically never press return or space more than once, and always have perfectly correctly functioning and formatted code at the end.

> So you could theoretically never press return or space more than once,

There are people that actually do this?

Oh, I thoroughly understand the parent's point, I'm just disagreeing that it is a problem that needs to be "addressed".

Most language design decisions involve tradeoffs, and for me this one has been a big positive, and the potential negatives have been - across decades of active usage - almost entirely theoretical.

Also, the real-world use case for importing code and auto formatting it is quite doable with Python - the main thing you can't do is blindly (unintelligently) merge two text buffers and expect it to work every time, but in general it takes only a smidgen of analysis on the incoming text to be able to auto reformat it to match your own. You could go all the way and parse and reformat it as needed, but much simpler/dumber approaches work just as well.

What you can't do is take invalid Python code and expect it to autoformat properly, but that borders on tautological, so... :)

Never commented out a loop or a condition, have you?

...yes? Sorry, I'm not sure I understand what you're getting at. :)

(funny enough, that particular scenario is actually harder to miss in Python since it often produces an error that prevents the program from running at all)

Ok, let's say I have some code. C:

  some_function();

Python:

  some_function()

I am debugging how this function works so I actually only want it to run when some condition is true. C:

  if (some_condition) {
  some_function();
  }

Python:

  if some_condition:
  some_function()

Oops. That's not indented correctly, so it won't run. To be fair neither actually looks good but that's 1. fine because this is for debugging and 2. for the C code I just format the file and it is instantly fixed. What if I have a loop? C:

  for (int i = 0; i < 5; ++i) {
      iteratively_optimize();
  }

Python:

  for i in range(5):
      iteratively_optimize()

Unfortunately this breaks something so I want to step it through. In C I comment out the loop as follows:

  // for (int i = 0; i < 5; ++i) {
      iteratively_optimize();
  //}

In Python:

  // for i in range(5):
      iteratively_optimize()

Nope, that's broken too. I can't just autoformat this code either because the formatter can't look at scope using anything else. I have to manually go and fix the indentation on that line too.

These are actually very small cases. I can even imagine you saying, in C you have to fix two places if you want to comment out a loop or if: the opening brace and the closing one. In Python you need to comment out the control statement and the second thing is fixing the indentation, so what's the big deal? Well, as the number of lines in the block grows larger in C it's still just commenting out the two braces, while in Python you have to select the whole region line-perfectly and fix the indentation. As someone who writes both I always find this to be a lot more fiddly and annoying.

Expecting formatters to fix your broken blocks is a false economy with Python and smart indentation. It takes fewer keystrokes to write Python correctly, than to add delimiters so you could write it incorrectly.

Python’s tradeoffs pay dividends every day, at the expense of few questions a year. Also code is read 10x more than written, where the extra delimiters lower signal to noise.

Maybe it would help to think of "enter" as "semicolon" (since it's the statement terminator) and configure a macro or shortcut in your editor to act as "end" which decreases the indentation level by one step.

Wouldn't that make it behave pretty much as what you expect?

Bracket free-ness is still my favorite part of python

is 3.13 bigger than 3.9

The ones saying they will never use Python because it’s slow, is the probability high that their “fast” language is not thread safe?

What do you mean by thread safe exactly? Instead of Python I would use Typescript or Rust or possibly Go.

Python isn't "thread safe", not even its lists are. What are you getting at?

Python is not thread safe.

How many teams actually decide against using Python because it's "slow"? I'll personally never really get why people prefer interpreted languages in the age of Go, but even if we go back a few years, you would just build the computation heavy parts of your Python in C. Just like you would do with your C#, your Java or whatever you use when it was required. Considering Instagram largely ran/run their back-end on Python, I'd argue that you can run whatever you want with Python.

Maybe I've just lived a sheltered life, but I've never heard speed being used as a serious argument against Python. Well, maybe on silly discussions where someone really disliked Python, but anyone who actually cares about efficiency is using C.

You occasionally hear stories about teams writing something in Python, then rewriting it in another language because Python turned out to be slow. I have one such story.

With the excellent Python/Rust interop there is now another great alternative to rewriting the heavy parts in C. But sometimes the performance sensitive part spans most of your program

Python is a lot like PHP. A uniquely bad value proposition in almost all aspects.

It is a slow interpreted language, but that isn't the only argument against it.

It has abandoned backwards compatiblity in the past and there are still annoying people harassing you with obsolete python versions.

The language and syntax still heavily lean towards imperative/procedural code styles and things like lambdas are a third class citizen syntax wise.

The strong reliance on C based extensions make CPython the only implementation that sees any usage.

CPython is a pain to deploy crossplatform, because you also need to get a C compiler to compile to all platforms.

The concept behind venv is another uniquely bad design decision. By default, python does the wrong thing and you have to go out of your way and learn a new tool to not mess up your system.

Then there are the countless half baked community attempts to fix python problems. Half baked, because they decide to randomly stop solving one crucial aspect and this gives room for dozens of other opportunistic developers to work on another incomplete solution.

It was always a mystery to me that there are people who would voluntarily subject themselves to python.

> How many teams actually decide against using Python because it's "slow"?

At least mine. Also because of the typing. It's probably improved, but I remember being very disappointed a few years ago when the bloody thing wouldn't correctly infer the type of zip(). And that's ignoring the things that'll violate the specified type when you interface with the outside world (APIs, databases).

> anyone who actually cares about efficiency is using C.

Python is so much slower than e.g. Go, Java, C#, etc. There's no need to use C to get a better performance. It's also very memory hungry, certainly in comparison to Go.

Except “like you would do with your C#, your Java…” does not happen w.r.t. native components - you just write faster code and in 98% situations it’s more than enough. Now, Java and C# are different between each other when it comes to reaching top end of performance (Java has better baseline, but C# can compete with C++ when optimized), but we’re talking about the level far above the performance ceiling of Go.

Most of us use Python, just not for stuff which needs to be fast.

Not sure, what you mean by thread-safe language, but one of the nice things about Java is that it made (safe) multi-threading comparatively easy.

With Rust, code runs at native speed, multithreading is easy and safe, and the package manager doesn't suck. I will never use Python if I can help it, but not just because it's slow.

Reply on news.ycombinator.com