Intent to approve PEP 703: making the GIL optional

Around that time, doing cross-platform C++, I got an early look at Java, with concurrency built in from the start, along with GC and various other nice features that were easier to use than C++, and I "knew" it was going to be huge. (But who knew that the MIS people would take over Java, when it seemed clearly targeted at non-MIS programmers, and now MIS people are stuck with the C++ syntax and verbosity, after coming from 4GLs, etc.)

Then mainstream programmers picked up Python, which, IIRC, originally was an embeddable extension language, which was why it was simple. And for which the GIL made more sense.

No one's flipping a switch and breaking mountains of sketchy C.

Even writing a JVM native JNI library would have allowed to avoid a lot of that pain (and the library would have been useable from Clojure, Kotlin , Scala, JRuby[1], Jython[2], Java, etc) without any painful threading issues.

[1] which I’m aware of having been used in production by companies in the past

[2] which I’m aware has been quite a bit under maintained for the last several years

In Python you have whole C modules with global state. Load 10 of them, add the interpreter complexity and soon enough no one knows what is going on any more.

As it is, most developers (including core devs!) don't even bother to check for memory leaks. I don't think they'll run tsan, and if they do, it will be on a small test suite that only covers 10% of the code.

Given the software development practices in Python and especially in the AI space, I'm very pessimistic about this feature.

We're now up to 128 core CPUs, and even cheap CPUs have 6 cores on them. Restricting things to a single core's performance gets more and more limiting as time passes.

I also suspect that the GIL has saved us from debugging reentrant and/or dangerously concurrent code for years, and I salute the GIL for forcing us to build Arrow for IPC in Python, in particular.

Someday, URI attributes in CPython docstrings might specify which functions are constant time, non re-entrant, functions.

Reentrancy (computing): https://en.wikipedia.org/wiki/Reentrancy_(computing)

Global Interpreter Lock: https://en.wikipedia.org/wiki/Global_interpreter_lock

Many libraries have never had a multi-processing use case, others are so big that bugs are bound to happen, many of them subtle, so one guaranteed outcome will be unreproducible complaints and devs throwing in the towel. Opt-in will make people unhappy.

For libraries that are reentrant but not thread safe, it should be sufficient to just add a global lock wrapping every call, which is pretty close to what the GIL was doing anyway.

It seems to me that in many (most?) cases it should be relatively straightforward to make existing libraries work without the GIL (at the cost of parallelism). I guess the main issue will be for libraries that call back into the python runtime from the C side.

Anything that benefits from both parallelism and replacing IPC overhead with shared data between parallel tasks.

Multiprocessing does the parallel computation pretty well as long as the granularity is not too small. When smaller chunks are needed most of the time that's something better done from an extension.

No-GIL multi-threading is multi-core, though difficult to use. I don't know the Python implementation but shared memory should be faster than using multiprocessing.

That said, when designing a system from scratch, I completely agree with you that for almost almost almost all Python use cases, threads should never be touched and asyncio/multiprocessing is the way to go instead. Most Python programs that need fast multi-threading instead should not have been written in Python. Still, we're here now and people did write CPU-intensive code in Python for one reason or another, so no-GIL is practical.

In these threads, I also always see a lot of people who simply aren't aware of asyncio/multiprocessing. I assume these are also a significant share of people asking for no-GIL, though probably not the ones pushing the change in the committee.

Threads let you use the same unified abstraction for parallelization and concurrency. They also make it easier to share state with parallelization (no need to go out of your way to do it) at the cost of requiring you to think about and implement thread safety when you do so.

Also, with no-GIL + threads the computational costs of creating and maintaining a parallel execution is much less vs multiprocessing. And data sharing and synchronization are less expensive.

What LMAX is doing is really just an overhyped way to speed up producer-consumer models. It might apply to your use case but it’s not the only reason you’d use parallelism or concurrency. I don’t even understand why they are claiming it to be an innovation when it’s just using a LockFreeQueue implementation within a pre-allocated arena? You also can’t synchronize with their implementation, which sometimes you really need to do. Not a silver bullet

1. Because asyncio is completely useless when the problem is CPU bound, as the event loop still runs only on a single core. As the name implies, it is really only helpful when problems are IO bound.

2. Because sharing data between multiple processes is a giant PITA. Controlling data AND orchestrating processes is an even bigger pain.

3. Processes are expensive, and due to the aforementioned pain of sharing data, greenlets are not really a viable solutions.

Multithreading is more efficient but more difficult to work with.

You share the same address space in threads, so you can communicate any amount of data between threads instantly within a lock. The same cannot be said for network traffic or OS pipes or multiprocessing.

Multiprocessing uses pickle to serialize your data and deserialize it in the other python interpreter.

If you start a Python Thread, you're still single threaded due to the GIL.

But I have the same question as you have if we add another coming concurrency model: SubinterpreterThreadPool, which will be possible with the per-interpreter GIL in python 3.12 and later.

That's another new model that is already confirmed to be coming: interpreters (isolated from each other) in the same process, that can run with each their own GIL.

I mean a situation like where in threaded context there would be code like:

    def determine_quest_latency(quest_name: str) -> int:
        return other_thread.wait_sync_job(lambda context: context.ping_quest(quest_name))

I believe the answer is "you don't", but passing functions in messages is a highly convenient way to structure code in a way that local decisions can stay local, instead of being interspersed around the codebase.

Tons:

- A web server that responds using shared state to multiple clients at the same time.

- multiprocessing uses pickle to send and receive data, which is a massive overhead in terms of performance. Let’s say you want to do some parallel computations on a data structure, and said structure is 1GB in memory. Multiprocessing won’t be able to deal with it with good performance.

- Another consequence of using pickle is that you can’t share all types of objects. To make matters worse, errors due to unpickable objects are really hard to debug in any non-trivial data structure. This means that sharing certain objects (specially those created by native libraries) can be impossible.

- Any processing where state needs to be shared during process execution is really hard to do with the multiprocessing module. For example, the Prometheus exporter for Flask, which only outputs some basic stats for response times/paths/etc, needs a weird hack with a temporary directory if you want to collect stats for all processes.

I could go on, honestly. But the GIL is a massive problem when trying to do parallelism in Python.

> Python’s global interpreter lock makes it difficult to use modern multi-core CPUs efficiently for many scientific and numeric computing applications. Heinrich Kuttler, Manuel Kroiss, and Paweł Jurgielewicz found that multi-threaded implementations in Python did not scale well for their tasks and that using multiple processes was not a suitable alternative.

> The scaling bottlenecks are not solely in core numeric tasks. Both Zachary DeVito and Paweł Jurgielewicz described challenges with coordination and communication in Python.

> Olivier Grisel, Ralf Gommers, and Zachary DeVito described how current workarounds for the GIL are “complex to maintain” and cause “lower developer productivity.” The GIL makes it more difficult to develop and maintain scientific and numeric computing libraries as well leading to library designs that are more difficult to use.

> Manuel Kroiss, software engineer at DeepMind on the reinforcement learning team, describes how the bottlenecks posed by the GIL lead to rewriting Python codebases in C++, making the code less accessible:

> "We frequently battle issues with the Python GIL at DeepMind. In many of our applications, we would like to run on the order of 50-100 threads per process. However, we often see that even with fewer than 10 threads the GIL becomes the bottleneck. To work around this problem, we sometimes use subprocesses, but in many cases the inter-process communication becomes too big of an overhead. To deal with the GIL, we usually end up translating large parts of our Python codebase into C++. This is undesirable because it makes the code less accessible to researchers."

For average usage like web apps, no-GIL can be solved by multiprocessing. But for AI workloads at huge scale like Google and DeepMind, the GIL really does limit their usage of Python (hence the need to translate to C++). This is also why Meta are willing to commit three engineer-years to making this happen: https://news.ycombinator.com/item?id=36643670

You can have shared-memory parallelism with near-zero synchronization overhead. Rust's rayon is an example. Take a big vector, chunk it into a few blocks, distribute the blocks across threads, let them work on it and then merge the results. Since the chunks are independent you don't need to lock accesses. The only cost you're paying is sending tasks across work queues. But that's still much cheaper than spawning a new process.

For creating stateless, mostly IO bound, servers, it's great. Try to squeeze in performance and it all starts to fall apart.

It just means for the brave few that flip it on and use it, be prepared to spend a huge amount of time finding and fixing subtle race conditions in decades of old python library code. The early adopters are going to be in for a lot of pain, or more likely they'll restrict their use of non-GIL processes to very specialized and dedicated processes that have as few dependencies as possible.

The argument against this seems mainly that it's a lot of work; not that it's impossible work. It probably is a lot of work but if there are enough people doing the work, we should get some results.

As someone who works professionally on a parallel / async runtime that supports thousands of continuously running servers, "extremely unlikely" means that actually it breaks all the time, but it's also impossible to debug.

But apparently Python needs some "success" and this makes a good bullet point. Correctness does not really matter in the Python world.

That is too long for Python to have 2 modes. Half a decade is more than enough time for 2 modes to become the status quo. For one thing, think of all the outdated Stackoverflow threads that will be hanging around after that 5 years. I am not optimistic that 5 years won't turn into 10 years of uncertainty and breakage.

But 5 years might be too short for everyone to dredge up all that C code, update it, test it, and call it mature.

I guess we'll see.

For any tool builder that means their cost has just doubled for the next five years, at least. Why? Because people will want to use tools in either mode, no matter if it is deemed productive or experimental.

1. We don’t want it to be.

2. We’ll give up quickly if it is.

Those are both important points. But there seems to be an important missing third piece of “and we’ll achieve this by…”.

A similar misunderstanding exists about SQLite and concurrency.. but that's a topic for another time.

In Python there's a much better option if you care about performance - use a different language!

IPC is a PITA, and orchestrating processes is even worse.

> or at the isolated numeric library level

Not everything I want to parallelise in python runs in numpy. Simple example: WebService Backends. I have a 64 core server running a Werkzeug/Gunicorn application. The Service is mostly doing CPU bound tasks (data aggregation and analysis), so asyncio is pointless.

What happens is, it runs 60 worker processes. Which puts hefty limitations on any crosstalk and data sharing, because these either require IPC, or using redis/sql. Which are nowhere near as performant as actually shared memory would be.

The website you're using likely benefits immensely from thread-safe code. The browser you're using benefits immensely from thread-safe code. Your entire user experience using the modern internet is in an ecosystem of thread-safe code which you apparently are completely oblivious to.

There are a lot of threading models besides Java and C's as well, which you're apparently also unaware of.

Why would you assume you know all the possible use cases of a multi-purpose programming language?

I’ve often thought that proper threading should be learned by the masses, not to be scared of it. This sort of statement does nothing but spread FUD.

If you don't care about CPU performance, instead of threads you should go for an event-loop approach (see asyncio in Python). As soon as you have threads (on a language-level, not implementation-level), there is some notion of implicit switching, and you run into issues. So, the language you're looking for is JavaScript, which is single-threaded and every context switch is explicit (in form of `await` or `yield`).

[1] https://verdagon.dev/blog/python-data-races

All of the multithreading bugs without any of the multithreading performance.

You should hear about async...

It's simply not "sane multithreading" not being able to run Python (byte)code concurrently. It's just a huge annoyance.

> because it might lose its beginner friendlyness

Python is not beginner friendly. It has one of the worst documentations out there. Also it does things very different compared to other languages (C#, PHP, Java, JS, ...) - I would advise anyone against learning Python as their first language, while Python is my favorite language.

> And Python already has an expert mode - called Cython oer Numba to name just two.

CPython is the "official" Python. How is it beneficial to depend on some 3rd party projects? I don't want to use such projects.

As to the PEP itself: It's optional, with the GIL being enabled by the default. So what's the problem...

First, I don't see how the GIL would have much to do with free multithreading. The GIL should not have much observable logical impact on multithreaded _Python_ code. It should not make it more or less susceptible to race conditions. It's only practical impact should be slowness.

Second, your proposed "one GIL per thread" is pretty much the equivalent of the current state of multiprocessing. In that you fork your current interpreter state in an other thread with it's own GIL and start from there. This has been used for decades already, nothing new there. Sharing can be done through queues or shared memory.

This is like Ruby's Ractors and I haven't really seen that be super successful so far. The "Objects" that need to be shared are things like class definitions, etc.... there are a ton of subtle issues with objects that are being marked as sharable when they should really not be or vice versa.

You just described Ractors in Ruby, which didn’t turn out great. The setup cost for either freezing or copying memory to the target ractor to guarantee memory safety is often higher than the perf gains of the parallelism.

Not that it can’t work or won’t be improved. But there is a real world case study of what you’re recommending that we can reference without having to guess.

>offer a safe path to concurrent programming

Uh, doesn’t Python support this already? Python has “concurrency” from async and parallelism from multiprocessing. What it doesn’t support is thread-based parallelism. What you’re suggesting (if I understand it) is an implementation of parallelism that is barely different at all from the typical Python parallelism approach of using multiprocessing to achieve parallelism.

From what I understand, this is how threading worked in perl. But that functionality is now "discouraged".

I do think that would have been a good way to do it. Especially with an emphasis on message passing.

Just like you can today spawn threading.Threads one by one in python, or just map a bunch of function calls over a threadpool using concurrent.futures thread pool.

https://discuss.python.org/t/a-fast-free-threading-python/27...

Why should Python even have a free threading model?

Please tell us: What other solution do you propose for running CPU bound workloads in parallel?

There are exactly 2: Multiprocessing and using another language.

I'm afraid we'll soon learn its not the version # that's burdensome, but the real or perceived(!) incompatibility between versions.

I wonder if introducing such a monumental change in a build flag of the minor version is really wise. Certainly its not in line with any interpretation of semantic versioning (to be fair I think the PSF does not claim to use that).

Like asyncio's introduction we'll probably see core stuff like http requests, file IO etc. all now have an entirely new permutation of libraries made to support non-GIL mode. This is going to get pretty spicy as stuff like http already has regular (blocking IO) and asyncio (non blocking IO) versions, so now do they need regular non-GIL and asyncio non-GIL versions too? Is the default for a library author going forward to be creating four permutations of your library with vastly different behavior in each of them? Yuck.

It’ll make the Python source code much more complex and complicated, which is probably not a big deal, though I’ll say the CPython source is quite brilliant.

It’ll also mean for C library developers that they can’t assume Python opcodes are atomic. But I’m not sure C library developers will really mind too much because they already worry about this kind of stuff.

The underlying pressure on the Python ecosystem is to transition to a post-Moore's law era and effectively become a HPC platform where the "same" code runs on a CPU, a GPU, multicore, clusters etc.

Python may feel the pressure more than others because of the GIL and the fact it is used in compute intensive tasks more than others.

But this major need to transition to easy and seamless HPC/heterogeneous computing is the same for all languages. The question is who will get there first.

https://peps.python.org/pep-0703/#performance

The old method — which is the GIL, or non-non-GIL — provides no thread safety to Python code. It only protects C code

> Can we get an in-python fork() mechanic ... please?

You probably want Multiple Subinterpreters: https://github.com/python/cpython/issues/84692

Non-GIL adds some complexity to the implementation and some risk when writing multithreaded code at the benefit of improving performance.

No-GIL means it is possible to run things in parallel (without resorting to fancy C extensions).

I _am_ a python programmer.

> What is the use case for this?

> Who needs it?

This can be answered with a simple user story:

As "THE PYTHON STEERING COUNCIL",

I want to "GET RID OF THE GIL",

so that "PEOPLE WILL STOP WHINGING ABOUT THE GIL."

To be fair, there's another group of people who stand to benefit. Namely, any python programmers who currently believe that "threads are an easy way to make my program go faster" will soon be the recipients of a valuable learning experience.