Hacker News Reader

If you have a multi-threaded program with data races, you already have a problem. The GIL does not mean no data races are possible. It just means that only one thread at a time can run Python bytecode. But the interpreter with the GIL can switch threads between bytecodes, and many Python operations, including built-in methods on built-in types that many people think of as "atomic", require multiple bytecodes. That's why Python already provides you with things like locks, mutexes, and semaphores, even though it currently has the GIL.

To put a finer point on this, I’ve had the misunderstanding in the past that the GIL made Python like JavaScript in some sense (only releasing the GIL on some explicit parts of code like sleep). But really Python threads can switch in the “middle” of your code. The reason the GIL is annoying is mostly performance related for Python code itself.

My understanding is the GIL does not protect against Python-side bugs, and bugs from GIL removal would only be introduced from C extensions.

? Why do you think this?

This has been discussed extensively in the past (1), and my understanding of the take away was that the GIL doesn't protect you from arbitrary execution order; it protects you from undefined behavior due to concurrent write/read in parallel scopes and the resulting data corruption.

...which, as I understand it, there is no specific reason it would be restricted to native extensions.

Is there some more detail to the nogil proposal that addresses the type of UB you see in eg. c, with this? (Wouldn't that require that at some level the GIL still exists?)

[1] - https://news.ycombinator.com/item?id=30420579

What you said is precisely what nogil work is about. It's about replacing one global lock with finer grained synchronization primitives without much performance regression.

> It's about replacing one global lock with finer grained synchronization primitives

Not really, no. The finer grained synchronization primitives are (a) already available in Python, and (b) necessary even with the GIL for reasons I've given elsewhere in this discussion.

What nogil does is enable multiple threads to run Python bytecode at the same time, so that CPU intensive operations in Python can be parallelized without having to use multiple processes. Python objects that are accessed from multiple threads will have to be guarded with synchronization primitives under some circumstances where, in principle, they don't need to be guarded now (operations that only take a single bytecode), but in practice I don't think that will make much difference. The big issue, as has been mentioned elsewhere in this discussion, is C extension modules.

> The finer grained synchronization primitives are (a) already available in Python, and (b) necessary even with the GIL for reasons I've given elsewhere in this discussion.

The finer grained synchronization primitives does not already available in Python. Or, it should not be visible to Python code at all. What I'm talking about is the internal implementation of, e.g. PyDict. While from Python bytecode side setitem on it is already not atomic, it does guarantee that Python interpreter won't segfault if there are two Python threads manipulating one dict object concurrently. This is achieved via GIL and has to be replaced.

It's the same problem you mentioned above as "problems in C extension". But no, nogil is hard not only because of compatibility issues. People (especially those who insist on that their workload is inherently embarrassingly parallel) do NOT accept any regression in single-thread performance. If you only ever want to optimize for single thread one global lock is the optimal solution.

You’re right that the GIL prevents bugs on clobbering exactly the same part of memory in Python. But in the GIL world, a C extensions method that doesn’t release the GIL and doesn’t call into Python has an extra guarantee that it won’t be interrupted at all. This means that in GIL-land, a C extension can have implicit critical sections that stop being so in noGIL land.

> You’re right that the GIL prevents bugs on clobbering exactly the same part of memory in Python.

No, it doesn't, except for operations that complete in a single byte code. See my response to shrimpx downthread.

Sorry, I'm being too handwave-y, what I meant (and what i assumed parent meant) is simply that those single bytecode operations are safe thanks to the GIL, so your dictionary or list isn't going to become corrupted because of simultaneous writes.

like l=[], then l.append(1) and l.append(2) running concurrently will not end up in some weird scenario where the length of the list is 1 yet you stored two items... anyways, I agree with the comment you posted higher up in the discussion, and that was my understanding.

> This has been discussed extensively in the past (1)

Yes, and I weighed in on similar lines in that discussion:

https://news.ycombinator.com/item?id=30423255

I think by “data races” the parent means that code doesn’t lock around operations like += and len(). The data races are there in theory but do not exhibit due to the GIL.

But this isn't actually true; that was my point. Neither of these operations, if they are part of an actual statement or expression, will complete in a single byte code. That means that, with the GIL, the operations can be interrupted with a thread switch between bytecodes. So, for example, a statement like

a += b

takes four bytecodes: two LOAD bytecodes to put the values of a and b on the stack, an INPLACE_ADD bytecode to put the result at the top of the stack, and a STORE bytecode to store the result in the variable a. A thread switch could take place in between any two of these bytecodes, and if another thread mutated a or b or was reading the value of a or b, you would have data corruption. The GIL does not prevent this. The only way to prevent it would be to use one of the locking mechanisms provided to guard access to the a and b variables so that only one thread could access them at a time.

Thank you, I believe you are right. I did try to make a race condition happen using += and am having a hard time. Here's my code

    import threading

    i = 0

    def test():
        global i
        for x in range(1000000):
            i += 1


    threads = [threading.Thread(target=test) for t in range(20)]
    for t in threads:
        t.start()

    for t in threads:
        t.join()

    assert i == 20000000, i

Presumably the assert should fail sometimes but I haven't been able to observe that.

CPython is not really in a good position to change behaviour in incompatible ways and defend it by language lawyering, users expect the same code to mostly work in a backwards compatible way.

Just a little fun fact: The GIL absolutely does not prevent all race condition bugs. Threads contending for the GIL can already steal it from each other at unfavorable times and cause havoc.

In fact, it prevents very few race condition bugs.

Even inside a C extension where the Python API feels like it gives you control over when you release the GIL (with functions you'd have to call explicitly to release the GIL), it turns out that:

* any operation that allocates new Python objects might trigger garbage collection

* garbage collection may run `__del__` of objects completely unrelated to the currently running C code

* `__del__` can be implemented in python, thus releasing the GIL between bytecode instructions

Thus there's a lot of (rarely exercised) potential for concurrency even in C extensions that don't explicitly release the GIL themselves. nogil will make it easier to trigger data race bugs, but many of them will already have been theoretically possible before.

I think the point was to let libraries declare whether they support nogil mode (opt-in), and your program would only run with no GIL if all the dependencies allow it? So they have all the time in the world to iron out those bugs.

At what point can an interpreter establish that a given python script will not be importing any more modules?

If you import another module the gil could be re-enabled.

Not saying that's what they will do, just what they could.

Right. It would be possible to implement the GIL as a readers-writer lock, where thread state includes a counter for the number of frames in the call stack that are within libraries not marked nogil. (Let's call these non-nogil libraries "GIL-dependent".)

When the count goes from zero to one, the thread attempts to upgrade its reader lock to a writer lock. When its count goes from one to zero, the thread downgrades its lock from writer to reader.

That way, there's at most one thread executing within GIL-dependent code at a time. Furthermore, if there is a thread executing within GIL-dependent code, all of the other threads are blocked waiting to acquire the GIL (in reader mode if they're nogil-safe, and writer mode if they're GIL-dependent.)

As now, any thread holding the GIL in writer mode would need to drop the GIL when attempting to acquire any other lock (and re-acquire immediately afterward).

To prevent starvation, one would presumably need a mechanism similar to periodic GC safepoints where nogil-safe threads still check if any thread is waiting to acquire the GIL in writer mode.

Perhaps it could just fail at runtime if you ever import a module that doesn't support nogil mode? AFAICT that's how it works if, for example, you run Python code that uses f-strings in a Python version that doesn't support f-strings.

Python's support for run-time version/flag/setting detection isn't great for this kind of thing either. With Perl, for example, you get a BEGIN {} block that is run before it tries to parse the script...so you can detect and shim, etc.

Python bombs before you can gain any control, because it parses the whole file. So you can separate things into modules to get around that, but it's not great when you want a simple one-file script.

We experienced programmers who know how to separate things into modules and use it wisely at the correct place. But most of the beginners and intermediate developers always tend to do it in a "Everything in single script" manner which might also make it difficult to have more control over the application's behaviour.

You're making a lot of assumptions in a smug sort of way. There's plenty of spots where a single script makes sense, and plenty where it doesn't.

If that is the only way. They need to change some semantics around import statements to not be runtime conditional. (Mypy can do it without running the code so to some extent it is possible). With that in place, each module can at the top declare #nogilsafe and the interpreter can know that no more modules will be loaded runtime. Dynamic imports via importlib will need other consideration.

Expecting every transitive module to add this marker is very optimistic though. It’s thousands of packages with hundreds of modules each, that need to add this everywhere.

Other languages have done similar journeys, like typescript “strict” added at top of each file. Except those are a lot more local, by not expecting all dependencies to follow.

I kind of thought Python would enable the GIL if any no-nogil libraries were installed, but even that is hard to define when you can modify sys.path with environment variables and code.

Will look it up.

Yes, but the plan is to remove the opt-in in time. That will put a lot of pressure on the eco system. I expect many libraries written in C or relying on C-based extensions to simply get dropped. Which will make that users will stay on the last GIL-supporting version. It's Python 2->3, but potentially worse.

In my important dependencies, there are deep learning frameworks from billion-dollar companies, bindings for C++ libraries that are basically standard in the field, projects from CS labs with millions of users. I don't see any of them getting abandoned. But I guess I can't judge how much work removing the GIL could take. The big projects tend to be well-written and well-maintained, for what it's worth.

Which projects do you have in mind that have a significant user base, are still maintained, and would be too costly to port for someone to do the effort?

The small ones, of which there are many more. Written for some specific purpose, half maintained, used in 1 or 100 projects. Those will suffer. Or perhaps worse, they get fixed, but wrongly, and introduce subtle bugs when your application is under a somewhat heavier load. Good luck finding the offenders. You might not even see a bug, only bad or irreproducible results.

Multi-threading, concurrency, and parallelism are fraught with problems. Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety. Certainly not when you can simply stick to Python 3.last-gil-version.

> Your precious ML/DL libraries may not even be upgraded, because writing neural network code is not the same as writing thread-safe code. If it comes from a CS lab, its authors have already left, and there's nothing worthy of a publication in adding thread-safety.

PyTorch and sklearn won't stop being maintained though. I don't rely on unmaintained research code in production, I adapt what I need under MIT license. Any other way sounds crazy.

Plus, most research code is very high-level and uses the same facilities (from e.g. PyTorch again) that everyone else uses, the actual distributed and multithreaded work happens in the main libraries. You'll still be able to use the same neural network code that worked before.

I don't see a huge problem for people who already had their dependency list under control. If you had anything that's both hard to replace and not big enough to be upgraded though, I'd argue that it was always going to bite you at some point.

I suppose no-GIL Python will be here in no less than 3/4 release cycles. 3.11 has been out for a year and most Python code in prod is what, 3.8? So I guess we won’t have to deal with this at scale before idk 2030 is approaching. I also don’t see Python runtimes in prod being updated from whatever they’re on now to newest releases. I don’t want to sound harsh, but the SC stated they don’t want to have another 2-to-3 migration, so people won’t update lightly. Yes, most of the content online right now might be dangerous to copy paste

Tangentially, 3.11 was the first release in quite some time to have major speed improvements across the board. The average is 25%, sometimes far more.

Anyone who hasn’t upgraded to it by now is needlessly spending extra on compute.

It's only 25% speedup in actual Python code, in domains like data science this won't matter much, because your cpu cycles are mostly spent inside numpy.

It does matter if you run your ML models in production. After we upgraded to 3.12 the average response time ~4-6ms decreased to stable sub 2ms. Latency decrease for p95 and p99 were even more significant

There are plenty of web apps running in pure Python. It matters a great deal for them.

> most Python code in prod is what, 3.8?

3.8 is the oldest supported version, so I would hope not, but probably.

Python versions I have to target (occasionally): 2.5, 2.7, 3.5, 3.8.

Not the whole world has the luxury to upgrade all their systems all the time.

Excuse me, would you mind stopping this factory for a few hours so we can replace this perfectly functioning system with an untested one that may or may not work in roughly the same way? is not a question that is generally met with wild enthusiasm.

The GIL only protect the Interpreter. The only thing it may do is to make it infrequently. There are MANY threading bugs in actual Python code.

> So much existing multithreaded Python code is written in an unsafe manner.

Even multi-process Python code is often broken. The "recommended" way to serve a Django app is to run multiple workers (processes) using gunicorn. If you point the default logs to a file, even with log rotation enabled, all workers will keep stepping over each other because nobody knows which file to use. Keep in mind that this is broken by default, and this is the recommended way to use all this.

On the other hand, perhaps translating existing modules to a No-GIL API is tedious but straightforward, and something that can be done using automated tools (perhaps even LLMs).

The easiest way would be to have the GIL behind a feature flag that defaults to 'on'. That way you avoid yet another language split and if you don't want any possibly breaking changes you simply don't do anything at all. But if you want to run with the performance gains that a GIL free CPython would give you then you will have to do some extra testing to make sure that your stuff really is bullet proof with that flag set to the 'No-GIL' position.

Even with nogil, libraries can explicitly hold a global lock before any call. They just don't have to. I imagine some libraries will do that, and others will target performance. Users will vote with their tomls

> how are they going to do this while making sure programs are still correct? So much existing multithreaded Python code is written in an unsafe manner.

a) It isn't a language maintainers job to make sure unsafe code written by someone else in that language runs correctly.

b) The GIL doesn't prevent data races. It keeps the internal running of the interpreter sane, that's it's only job. There is a reason the threading library has a plethora of lock-classes

There is a very simple practical solution: let the GIL (or something like it that forces single-threaded execution) be an option that you can turn on so that you can run broken legacy code.

Wouldn't the most practical solution be for a script or module to turn it off instead? Then you don't break any legacy code. Anyone writing code that is meant to work without the GIL would know to turn it off.

Sure, whatever. The point is that breaking legacy code doesn't have to be a show-stopper for eliminating the GIL, and backwards-compatibility does not have to be (indeed should not be) an overriding consideration in a GIL-free Python.

What's the rationale? Making it opt-in avoids breaking legacy code, which is a huge advantage. What's the caae making it opt-out?

Why are you asking me to defend a position that I have explicitly disclaimed?

No, you claimed the opposite. You suggest that disabling the GIL should be opt-out (or conversely enabling the GIL should be opt-in). This means that many legacy packages will break by default. You said it should be this way. Why?

Don't downvote me, answer the question.

Do you see where I wrote, "Sure, whatever." upstream? What do you think that means?

I saw the "indeed should not be", which seems to even more strongly indicate the opposite.

You seem to have some serious reading comprehension issues. The context of that phrase was this sentence:

"The point is that breaking legacy code doesn't have to be a show-stopper for eliminating the GIL, and backwards-compatibility does not have to be (indeed should not be) an overriding consideration in a GIL-free Python."

That has nothing to do with opt-in vs opt-out.

This is, effectively, the plan for the next few Python releases. The plan is for a no-GIL and a GIL execution mode with GIL being the default. At some point, IIRC, the plan is to swap the default and then to eliminate the GIL option.

It'd make sense if pypi wasn't full of abandoned libraries that nobody can reclaim to update.

I imagine the way to do this is to start Python with some flag saying that it's in no-GIL mode. That way it's up to the user to decide if their libraries can handle it.

Good points. Re analyzability - it wouldn't have to be static analysis to be useful, you could do this with dynamic analysis.