GIL removal and the Faster CPython project
lwn.net
lwn.net
It doesn't highlight enough the other side of the coin:
- Sam's work is very high quality, and he brought with the no-gil some unrelated perf improvements so that people don't feel like they loose perf too much.
- Sam played the open source game perfectly, and is incredibly patient given what he is bringing on the able and how slow and flaccid the steering council reaction was (without the community pushing on it, it would still be collecting dust).
- Sub interpreters have yet to demonstrate any usefulness at all in Python. In fact, any serious metrics at all. This is the first attempt to be that well defined, and measured.
- The community feedback shown a great interest in this particular project.
- The steering council did conclude "We intend to accept PEP 703, although we’re still working on the acceptance details."
I'm not a no-gil enthusiast. I would be fine with have it never be removed, and I think we should try sub-interpreter first.
But what's fair is fair.
There is a lot of work going on with sub-interpreters and the per-interpreter GIL is shipped in Python 3.12.
The results are very impressive: https://lwn.net/SubscriberLink/941090/8bcb029dbf548f26/, as good as one could have hoped I think.
It seems to me like the work on sub-interpreters will continue in parallel ;) to the work on free-threading.
Sub-interpreters and no-GIL have different use-cases though.
Now that the work from Eric Snow has been merged, you can use https://pypi.org/project/interpreters-3-12/ to create one from Python code.
Which means there is nothing subinterpreters do that can't be done with multiprocessing.
They show progress, and it's good. But I would wait until we can see gunicorn spawning WSGI interpreters workers and get similar performances to the setup with regular workers to get enthusiastic.
They use unmaintained C extensions that won't be updated.
They maintain C extensions that would be complex or painful to update.
They're concerned about fragmenting the ecosystem into GIL and No GIL.
They think that it will make single-threaded Python programs slower.
They have no interest in multithreaded Python and don't think the additional complexity is justified.
Well, that's on them.
> They're concerned about fragmenting the ecosystem into GIL and No GIL.
Same way we have fragmentated ecosystem into ASYNC vs SYNC now? /s
> They think that it will make single-threaded Python programs slower.
Python is already horribly slow. A little bit slower single thread programs shouldn't be a problem. But the inability to run multi-threaded programs effectively in this day and age is very much a problem.
I believe PDM still supports it. I just hate virtualenvs for builds and deployments and wish Python could just do what JS does. It’s been proven that it can. https://discuss.python.org/t/pep-582-python-local-packages-d... Is the discussion thread. Also frustrating that “exactly one correct way to do something” seems to be one of the justifications thrown around for rejecting this when I’ve never found that to be true in Python.
My experience is with writing backend systems in several different languages (including Python) at various volume/latency/throughput levels. I've basically worked on only two types of systems -
1. one that exposes some sort of an endpoint to the network - it accepts requests of some kind, does computation and other network requests and sends response of some kind (including long polling, ws etc).
2. reads a message from a "queue" (could be database, could be based on polling another api etc) and does computation/network calls and basically sends it to other queues.
Nothing else. Huge variance in specific requirements, but that's it. For the first type of system, latency matters more. For the second system, throughput matters more.
For the first type of system, I want to be able to spin up threads in response to requests, without worrying that an endpoint is too computationally heavy and might block others. I want to be able to share connections to databases in a shared pool. NoGIL would be useful here.
For the second type of system, I can't remember the last time where I wrote one where I had in-process parallelism/concurrency with shared resources (even in langs where there's no GIL). It would just get too confusing and hard to reason about. Any optimizations were mostly based on intelligent batching. For parallelism, you'd just have multiple _completely_ independent processes, probably across multiple machines.
I would absolutely be disappointed if NoGIL meant compromising on the quality of of the second type of system here. In practice, most of my mental bandwidth today goes towards making the second type of system better.
And that will raise everyone's code.
For myself, the prospect of no-gil is interesting, in that something like my Captain's Log application [0] can be free from it; for example, I currently use a QThread to implement a JournalParser, which is basically the program's "engine" - the parser constantly reads in game events from a player journal file generated by the game Elite: Dangerous (and Odyssey), and depending on the particular event, fires off a related custom QSignal, which is then processed by whichever slot (receiving function) is listening for a given Signal.
There are other places in that application where no GIL might be quite handy.
In other words, I can see where having no GIL can be useful for GUI applications like mine.
Also, in many ways, GUI applications written in Python are not so much CPU bound, but Python GIL bound. If you're writing a Python/Qt application, you have to take great care to ensure your GUI doesn't freeze when your program is performing, say, many database inserts; if you have some naive loop which performs some given operation, your nice Qt GUI will freeze right up until that operation is complete. Right now the solution is to perform such operations in, say, a QThread, and use Qt's signal/slot feature to blat a progress "report" to a handler in the `main` Python loop.
So back to what I said - no-GIL is looking quite interesting to me. Whether or not Qt can take advantage of such will be a different matter.
Of course, one may argue that you probably should not write CPU bound programs in Python in the first place, but that's another story :)
The GIL is not about the CPU but about enabling those kinds of things. With the current GIL in place it's very simple: as soon as you hit the global lock, everything stops until it is released. It doesn't matter how many CPU cores you have, they'll be idling while one of them holds the lock. There's barely any point in even trying to do that with the GIL in place. Forget about sharing data between threads. Mostly that's done via queues or databases in python. Removing the GIL will revolutionize a few things in key use cases for python:
- data processing & ETL
- event driven server systems
- machine learning and data science systems
They can all benefit from this and that's the reason a lot of people are pushing for this. The short term performance losses are not inherent to removing the GIL but just a necessary evil while the python developers deal with fixing the bottlenecks and a few decades worth of technical debt.
However, you are certainly right that not all server applications are I/O bound. I was a bit sloppy there.
Interestingly I'm working on something like this right now and do have large shared resources which meant I had to abandon using a multiprocess strategy.
I don't see why it would be confusing though, provided the shared resources are read only.
A couple of years ago, I implemented in process parallelism for a system I was maintaining at $JOB. I was happy the system was in Go and not Python. But it was an exception to the rule in my experience.
> I might be good to remind the readers that the GIL removal has very little chance to break a Python-only codebase
Is this actually true? I was under the impression that some multi-threaded Python code relies on some operations being implicitly thread safe due to the GIL. For example, adding an item to the same list from two concurrent threads is never going to corrupt the list, simply because the threads never do that operation in parallel (the GIL prevents the threads from running in parallel). If you remove the GIL, suddenly you'll punish this kind of code, just like C++ quickly punishes concurrent mutation of an std::vector.
I'm not 100% sure of this, but I find this sentence a bit suspect.
[1] https://discuss.python.org/t/pep-703-making-the-global-inter...
PEP 703 goes over this in the "Container Thread-Safety" (I think container here refers to the fact that the object has references to other objects, this is the things that already are special-cased in CPython to be managed specifically by the garbage collector):
> This PEP proposes using per-object locks to provide many of the same protections that the GIL provides. For example, every list, dictionary, and set will have an associated lightweight lock. All operations that modify the object must hold the object’s lock. Most operations that read from the object should acquire the object’s lock as well; the few read operations that can proceed without holding a lock are described below.
More information at https://peps.python.org/pep-0703/#container-thread-safety
But with it removed, you'll have to think about it in your designs more than currently. History shows us this is not easy.
Not really. The GIL is taken and released quite infrequently (only when the Python interpreter decides it's time to do a context switch), whilst the new locks for each data structure are taken/released every time you do a basic operation on those data structures.
Holding a lock that is rarely taken/released incurs very little overhead.
For example, Java implements different versions of containers for single thread and multithread usage, because multithreaded containers have obvious performance penalty
https://docs.oracle.com/javase/8/docs/api/java/util/concurre...
Very few codebases in Java are single-threaded; specialty frameworks like Netty are the exception not the rule.
Likewise, there are not different containers for single threaded and multithreaded usage; there are containers that have different strategies for dealing with multi-threaded usage.
Hashtable is the oldest and is notable for it still being present and still being fundamentally flawed. It will lock reads and writes, but does not describe a way to lock for a transaction - e.g. changes based on read data. As such, it fundamentally has race conditions you can't protect against.
Hashmap and the rest of the Java 1.2 collections API set a slightly better pattern - they don't internally try to maintain safety, but provide mechanisms like synchronizedMap() to let you hold the monitor for the length of your transaction.
However, this could only be so good, because the monitors in Java are pretty fundamentally broken as well. A monitor is both part of its public API ( e.g. "synchronized(foo) {...}" ) and part of its implementation (e.g. public synchronized void bar() { ... })". This means that external code can affect your internal operation if you leverage the monitor that you get by default through your "this" instance.
As such, synchronized set involves three monitors:
1. The monitor on the interface-implementing collection type itself, e.g. on the HashMap. This likely is never used.
2. The monitor on the object returned by the 'synchronizedXXX' wrapping method. This is used to protect transactional access, such as iterating through while removing items.
3. The monitor used as a mutex inside the object returned by the 'syncronizedXXX' wrapping method, protect the integrity of the collection data type if used by multithreaded code which does not hold monitor #2. The code may have a race condition, but it won't put the collection itself into an inconsistent structural state.
The 'synchronizedXXX'-returned wrapper objects are pretty expensive, and if you can you should just internalize those collections into business object that does any needed syncronization itself.
ConcurrentHashMap and the like are lockless, and are built with the idea that you can perform the changes needed through atomic operations rather than transactions. This isn't always true, but often is.
For a collection which is always held by a single thread, the atomic operation overhead may still cause a performance impact - after all, the atomic operations are still processor state synchronization points. It is also possible to beat ConcurrentHashMap with regular HashMap on certain usage in multithreaded environments, when you are properly protecting access to the HashMap yourself.
It might be challenging to find scenarios where ConcurrentHashMap doesn't beat the 'synchronizedMap()' wrapper, just because the implementation itself is really expensive.
not sure there's much they can do about this, other than protecting all the built-in data structure operations with mutexes, like java's original data structures (Hashtable, Vector, etc)
(but then how do you get a non-synchronized [] if you want one?)
Maybe it could be part of a non_threadsafe_containers module on Pypi.
(admittedly it's python so it's so slow it's probably not even measurable)
I'd assume that importing any sort of module level variable would imply an increment of the counter, but unsure.
> (but then how do you get a non-synchronized [] if you want one?)
You don't. This is one of the reasons why using the GIL is higher performance for single-threaded use-cases: stuff like lists and dicts can be non-synchronized
However - this is fundamentally the incorrect approach, because Vector and Hashtable aren't protected from read-then-write race conditions.
Such internal locking guarantees that the collection stays structurally sound, but not that code accessing it is dealing with a single consistent state until it finishes.
It's not remotely a trivial problem, and I assure you just about every naive solution has been considered and rejected.
The decision is entirely reasonable, to move forward in test mode, treat multiple interpreters as an interim experiment, but target being concurrent. The constraint of running old and new code in the same VM is a tall one. Surprisingly, LWN's summary says nothing about testing, which is largely unsolved and could lead to releases with unknown but serious bugs. Microsoft, Facebook/Meta, and Conda have stepped up with resources and a super-majority of core contributors want to move forward, but it's unclear what happens if things get hard and more people are needed.
Meanwhile a crazy number of projects in academia and industry from web sites to big data to AI depend on Python. The potential to export costs to python developers might be measured in percentage of GDP.
It doesn't sound like the problems are even known yet. It might be fairer to commit to the Faster CPython approach of knowable improvement over the next 3+ years, but have the concurrent Python promotors do more than just prototype. They should analyze the kinds of problems that could present, how they could be detected, and what can be done about it. This should be reviewed by people with backgrounds in proving concurrency guarantees. Then the questions can be fairly presented to the steering committee, when the unknowns are at least identified.
There's not a lot of program-management scale decision-making in open-source, but most have been relatively simple questions of driving the market in a given direction (apache, eclipse, linux). This has real technical unknowns.
And I can't help feeling they should also address inter-language ABI's at the same time. A big issue is matching the expected execution model of C. Java's foreign-function and memory interfaces have been incubating for many years, and Swift is also getting better at wrapping C and C++, but FFI's are notoriously (and likely unnecessarily) difficult.
Of course, that's also true of the benefits.
It kind of seems like Python suffers from insecurity / inferiority complex about GIL. I would instead go with the "JavaScript route" of fully embracing the single thread model. Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway. Being somewhat specialized and not supporting every use case isn't necessarily bad.
The JavaScript model is, basically, the sub-interpreter model. You have multiple independent heaps. Given independent heaps, you write single-threaded code and don't worry about locking, because no other thread can modify objects in your heap anyway, outside of special objects like SharedArrayBuffer.
Maybe you could enable GIL-free, single-threaded, sub-interpreter Python as a compile-time option. But it would break such a large amount of code out there. It would be a very difficult transition, practically speaking.
> Yes, some applications will remain hard / impossible, but I'd argue Python is not a good fit for apps requiring high performance / high scalability anyway
People are already trying to use Python for parallel tasks. Forcing people to use another language does not help them much.
It's way too late to artificially limit Python's reach like this. It's already heavily used in data science and machine learning. People need performance right now for things that they are already doing in Python. We can't go back to a time when that isn't true. Now it's one of the most common use cases for Python. People are building careers on this.
So yes, you can do Matmuls to your heart's content in one python thread and wait for file i/o in another thread without ever running into issues with the GIL. I'd even say if your Python Interpreter isn't bored waiting for stuff to happen, you're doing it wrong.
This is not true. The primary funding and motivation for the GIL removal work comes from the numerical computing community. The PEP (https://peps.python.org/pep-0703/) contains direct quotes from folks working on numpy, scipy, PyTorch, scikit-learn, etc. and also practitioners from places like Meta, DeepMind and so on, describing the practical constraints that the GIL places on many workloads.
Personally, I wouldn't be surprised all this work gets done on GIL-removal, and the end result is that if you run 10 threads in Python you max out at roughly a 3x performance improvement in your pure Python code (and often get less, I do mean that as a max not an average), with all the memory traffic it is doing. Honestly even if it did perfectly parallelize up to 32 cores, a completely impossible absurdity, it would still be noticeably inferior performance to what many languages will give you on one core. I can't help but think that if you're sitting here waiting with bated breath for the GIL-ectomy to improve your Python performance that that is a sign you should be rewriting your code right now in any number of languages that are simply faster. I very, very strongly suspect that this is going to result in very very disappointing speedups when it is done.
The no-gil project has already been benchmarked to scale basically linearly in this scenario, so your hand-waving predictions are already known to be inaccurate. https://www.backblaze.com/blog/python-gil-vs-nogil-boost-i-o...
Again, if you're sitting on slow Python code and hoping for the GIL-ectomy to save you, you should just go write it in a different language right now. A perfectly parallel Python will still be slower on a 32 core machine than other languages can be on a single core... and by some margin!
The competition I'm benchmarking Python's performance work isn't against other Python approaches, it's against other languages entirely. Subinterpreters are just a concession to the enormous pile of other preconditions Python (and dynamic scripting languages) brings with it and have no bearing on how other languages work; they're a hack around that problem. They're not a solution any other language would consider without those requirements weighing them down.
With multithreaded performance, you can just add another core to (more than) offset whatever overheads there are from using process-based parallelism.
I think that this entire GIL vs No-GIL dichotomy is misguided. The biggest problem people have with multiprocessing is that you can’t share memory. So add virtual processes with an explicit mechanism for memory sharing. Then you can keep all of your single-threaded optimizations like refcounting without barriers because the objects for one thread will stay in that thread.
If you need concurrency at the moment, you have already switched to using multiprocessing, so having a no-GIL multithreading is useless.
The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal.
Fixing _that_ should be the objective. What Python need is better support for placing native instances in shared memory.
> The only issue with Python/multiprocessing, is that sometimes you don't want queues, but shared mutable state. And as you said, placing Python objects in shared memory at the moment is convoluted, restrictive, and suboptimal.
The PEP goes into the motivation behind this work, and using multiple process does not magically solves all the issues:
> Multiprocessing, with communication via shared memory or UNIX sockets, adds much complexity and in effect rules out interacting with CUDA from different workers, severely restricting the design space.
> I reimplemented parts of HMMER, a standard method for multiple-sequence alignment. I chose this method because it stresses both single-thread performance (scoring) and multi-threaded performance (searching a database of sequences). The GIL became the bottleneck when using only eight threads. This is a method where the current popular implementations rely on 64 or even 128 threads per process. I tried moving to subprocesses but was blocked by the prohibitive IPC costs.
> NumPy does release the GIL in its inner loops (which do the heavy lifting), but that is not nearly enough. NumPy doesn’t offer a solution to utilize all CPU cores of a single machine well, and instead leaves that to Dask and other multiprocessing solutions. Those aren’t very efficient and are also more clumsy to use. That clumsiness comes mainly in the extra abstractions and layers the users need to concern themselves with when using, e.g., dask.array which wraps numpy.ndarray. It also shows up in oversubscription issues that the user must explicitly be aware of and manage via either environment variables or a third package, threadpoolctl. The main reason is that NumPy calls into BLAS for linear algebra - and those calls it has no control over, they do use all cores by default via either pthreads or OpenMP.
and it discusses the alternatives at https://peps.python.org/pep-0703/#alternatives.
You don’t need to share objects. Have explicitly shared buffers instead. Python is a dynamic language so you can easily build proxy objects that are views into a shared buffer, and this allows you to keep all your single threaded performance because no objects are shared.
For example:
buf = sharedbytes.alloc(1 * GiB)
with buf.lock(lockid):
buf[10:20] = 42
message_other_process(buf.id)
# other process
def recv_buffer(bufid):
buf = sharedbytes.get(bufid)
with buf.lock(lockid):
print(buf[10:20])
Wrong. Processes pose hurdles via the limits of IPC and control between processes that Threaded applications don't have to bother with. There are ample examples for this in the PEP.
> What Python need is better support for placing native instances in shared memory.
If that were the case, then threading wouldn't exist in the first place. Shared memory is still IPC. It still requires context switching into the Kernel. It still poses problems that threads don't have.
> It still requires context switching into the Kernel.
There is no less context switching between two threads than between two processes.
> It still poses problems that threads don't have.
Hu, no, it's actually the same problem. If you see threads as different processes with heap being mmap'd at the same location, then you're basically 99.9% right, up to some minor process accounting metadata differences.
I am not talking about switching context between the threads of execution. I am talking about a context switch to kernel code simply to access the shared memory. All interactions with SM require syscalls.
So no, I am not "talking nonsense", and yes, communication between multiple processes requires more context switches than communication between threads.
Please enlighten me with the syscall you use to access a shared memory... because there are none.
As far as the kernel is concerned, "memory" is just a set of pages mapped to a process at a specific address. These pages can be anonymous or named (meaning other processes can map them through that name).
There is no syscall, no context switch, involved to read or write memory, named or anonymous. Hell, that's even the whole purpose of memory mapped IO.
As mentioned earlier, there is also no real difference between a thread and a process as far as the kernel is concerned. A thread is just a special case of process which maps its heap to its parent.
Even the word "thread" doesn't really exist for the Linux kernel. We just call these "lightweight processes", because these are just processes with a shared heap.
Oh rly?
https://man7.org/linux/man-pages/man2/shmat.2.html
https://man7.org/linux/man-pages/man2/shmctl.2.html
https://man7.org/linux/man-pages/man2/shmdt.2.html
https://man7.org/linux/man-pages/man2/shmget.2.html
Pretty sure it says something about "System Calls Manual" at the top of all these man pages ;-)
And IO on the SHM once it's attached may look "free", but it isn't. The M-mapping incurs a further overhead which simply doesn't exist for threads: A threads heap space is the same address space as those of it's siblings in the same process.
1) You linked the man pages of sys5 shared memories. Everyone switched to posix shared memories since literally 20 years.
2) Furthermore, these man pages dont even support your argument. Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory.
I would suggest, for your own future development, that you care less about trying to look knowledgeable on internet, and more about actually being.
Which is irrelevant, because there is barely any functional difference between the two. POSIX SHM uses a better API, providing a file-descriptor like object. That's all.
And yes, using that API also requires syscalls.
> Furthermore, these man pages dont even support your argument.
Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup. And this requires syscalls, in SysV as it does in POSIX.
> Did you actually read them? These are just for the mapping, not for actually accessing (read/write) the shared memory
I am well aware of that, hence my seperate mentioning of IO later in my post. ;-)
And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace.
Now, there is a way around that, in principle: If I mmap with ANONYMOUS and SHARED, and then `fork()`, I could have the same mapping in the child process. The problem here is: This relies on the forking after the mmap(). Any newly created objects, if I manage to map them into the child process, will again require address mapping. Again, this isn't an issue in threads. As soon as I create a new object on the heap, it is available to every thread, under the same address.
But hey, what do I know. But I think the people who are going to dedicate countless hours of their lives making actual parallel processing via multithreading possible in Python know why they are doing so. As do the people who implemented abstractions around threading in basically every major programming language ;-)
Well it is relevant, it tells that you're not familiar with the subject and most likely googled what looked related and found outdated man pages which seemed related.
> Wrong, they absolutely do. "Accessing" something doesn't just involve the IO, it also involves the setup
I think common sense would disagree, but if that's your line of arguing, sure.
> And the argument in that post stands as solid as it was before. `mmap` MAPS memory. This mapping requires an address translation overhead EVERYTIME THE MEMORY IS ACCESSED. This translation doesn't happen in userspace.
How deep are you willing to dig your hole there?
Two messages ago you were arguing that accessing a shared memory required syscall and thus was slow. Now you've accepted it doesn't require syscall but argue it requires some magical dynamic translation by the kernel?
I'm sorry but again that is just wrong. All the kernel does is inject the mappings when setting up TLB/MMU at context switch.
The virtual address translation happens without kernel intervention, through the MMU, in _exactly_ the same manner whether this is a named or anonymous memory mapping, whether it is a shared or heap page. In fact the MMU couldn't care less what these addresses are, it has no concept of the meaning of what it translates.
That is an assumption, for which I have yet to see proof :-)
> I think common sense would disagree
Please, do enlighten me: How do I access SHM without mapping it first, or otherwise setting it up?
> Now you've accepted it doesn't require syscall
There was nothing to accept. If you actually read my post, you will see that I talked about the issues of IO and setup seperately.
And my point stands still solid ;-) SHM requires syscalls not required when directly using shared memory.
A lot of the performance reduction in python without the Gil is basically just unaddressed technical debt. That should be fixable over time. Adding a lot of locks is a stop gap solution. That indeed makes things slower. But the proper fix is probably rethinking how that stuff works internally in a lot of places or having API contracts that document the thread safety or lack thereof.
And also having faster python runtimes and compilers actually enables re-implementing a lot of things that currently depend on native libraries in python. A lot of native code interactions are precisely where you need locks. Unless you change how that works. The point of removing the GIL is getting systematic about finding and fixing those things. It will get better over time.
It depends what you're doing. Classic example: I have a huge dataframe, 10s-100s of GB. I want to process it in multiple threads, with each thread handling a different part of it. If I just use multiprocessing, it has to copy the memory to the new processes and then copy the results back, which is super slow. Sure there are hacky work-arounds and alternative approaches, but in a language without the GIL I don't need to fiddle around with those, I can just do the equivalent of Pool.map() and it doesn't need to copy anything.
2) There is no more copy in multiprocessing that in multithreading, to the very minor exception of the reference counting structures used internally by the interpreter which will get copied on write.
3) The problem you describe is not related to multiprocessing or multithreading, it's just related to a misunderstanding of the Pool.map() API. What you provide to Pool.map() is sent over a queue (and thus pickled) to worker threads - or processes. You don't _have_ to use this queue, so long that your function has a way to access the variable you want to use. That is, the following code will have your subprocesses share, without copying, your 100GB dataframe:
DF = pd.read_parquet('100GB.pq')
def worker_process(id):
# Do something with DF
return 42
with multiprocessing.Pool(10):
pool.map(work_process, [1, 2, 3, 4, 5])So, let me get this straight, some of those working in the project to "make CPython faster", think it's acceptable to overnight make most existing Python code 15-20% slower?
I'd say max 5% and that if the gil removal was a benefit to other optimizations going forward (they say the opposite: the change with complicate and stall their other optimizations).
Meanwhile, Shannon had a fundraising proposal for him personally "speeding up CPython" 5x back in 2020. Now it's a whole team working on speeding up CPython with much larger corporate support, and it seems their targets are quite smaller?
And for those that do not use multiprocess, rewriting their currently single threaded Python code for no-gil shared-concurrency will indeed bring a speedup, but would still be a significant rewrite.
Also there are people using Python for things like web serving, where performance might have been secondary to convenience when they picked Python, but still a sudden 15-20% slowdown will seriously impact their server budgets (or ability to update versions)...
Then why do we care about GIL removal at all?
That's the level of 2.7.
I seems that the old boys are desperate for some bullet point features for the next release to impress their corporate masters. So they use the work of Sam Gross, but they will slowly get the credit over time.
In a PL-fantasy-league they would just heavily sponsor Lars Bak to create a team and work on a new Python interpreter!
As for your fantasy, we already have PyPy.
The initial selling point was ~ 5x speedup over 5 years.
This is over a year now, with far more people and resources plus the backling of a major industry-player (MS), and it's been like 20%-30% improvements at best, and even those in danger to be wiped out by the no-gil change.
>As for your fantasy, we already have PyPy.
That's nowhere near Lars Bak level fantasy results...
As you said we see 20-30% at best. And several important workloads have important slowdowns (for example coveragepy runs 30-50% slower).
The GIL is an academic problem which has little real world relevance.
Some of the recent performance improvements came out of the no-gil camp to show that there was significant room for improvement even with the gil removed.
> (they say the opposite: the change with complicate and stall their other optimizations).
So they wont actually be stopped, it will just take them a bit longer?
I love the Python community. It's really a leading light for open source software. And it shows what transparency and good governance can achieve.
Although I appreciate the engineering hours that Meta, Microsoft and others give, its pretty miserable still compared to the value that the whole tech industry (and beyond with data science) extract from Python and other open source software.
I did my bit at JPMorgan 8 years ago, convincing the tech leadership team to sponsor PyCon UK, plus a recruiting stand and supporting a group of junior developers from all JPMorgan's UK locations to attend. I left JPM 5 years ago now, and they are still PyCon UK's headline sponsor.
Compared with the enormous benefit we got from Python and its open source ecosystem, it was a totally negligible cost.
Real contributors are exploited by those who work for the right corporations, do very little and go for any clerical position of power.
Do not be misled by the LWN articles, which are very kind and always namedrop the deciders. It is selective reporting.
Well, HN isn't censored in this regard (unless you believe their influence extends here too). What specific criticisms do you have against "the inner circle"? And what evidence can you show for them?
It is still a long and winding path (double-digit engineer-years) to get a no-gil python, but at least there exists a proper path from the looks of it.
The hardest part by far will be ensuring the correctness of all existing codebases. It's one thing to say you don't want a 2->3 repeat. It's another altogether if you were to claim no breaking changes, but fear of bugs resulted in folks avoiding the upgrade in practice.
Making gil/no-gil even a compile-time switch will absolutely increase the maintenance cost. But I think in the long run, all this effort will be worth it, as I would claim the GIL is a lightning rod for python criticism. Just peruse any HN thread about python and parallelism to see what I mean. Maybe it's because it's the one thing folks can point directly to and say "this is why python isn't as fast as it could be" without understanding the decades of context. It's kind of the Final Boss of Chesterton's Fences in that regard.
[0] https://github.com/python/steering-council/issues/188#issuec...
I am a beginner in this space but my hobby is paralellism, coroutines, async and multithreading.
I read about the subinterpreter approach and I think the subinterpreter approach causes me to think of actors. You destroy the reference in the source interpreter when you send data to the other interpreter, like message passing. But it should be a O(1) message passing and transfer of ref counting responsibilities. The interpreters would be separate so you wouldn't have to worry about a dictionary being updated in two threads because they never touch the same objects.
After reading https://blog.redplanetlabs.com/2023/08/15/how-we-reduced-the... I am thinking about how to parallelise the creation of relationships.
What's a good practice for parallelising behaviour? If I want to fan out a message to 20 million collections, how can I use paralellisation to make it faster?
There have been several attempts at integrating a JIT into CPython. Google made a stab at it back around version 2.6 and there was a second attempt around the 2->3 change to build a JIT on top of LLVM. However none of these projects produced results that the core developers felt were good enough across the board and had problems with backwards compatibility and lot cross platform LLVM issues (as it was a pretty new project) so they were dropped after Google stopped funding the project.
That being said there is still work being done and people are hoping for at least some JITing in upcoming python. Pyston is project being worked on Guido Van Possum himself and both Microsoft Instagram have their own JITed versions of Python they use internally. Some initial parts of Pyston are even scheduled for being included in python 3.12 or 3.13
The biggest one would be not using reference counting, which makes execution of cleanup code in __del__ somewhat non deterministic and tricks that rely on the exact reference count of an object outright impossible (for example reusing "immutable" objects that are only referenced once).
I thought that the C-API still had significant differences but going by its documentation it has a compatibility layer that is almost complete if somewhat slow.
A full list with all the minor differences is here: https://doc.pypy.org/en/latest/cpython_differences.html
I hope to see something akin to the Project Loom approach in Java whereby suitable parallelisation abstractions are native, and one configures concrete implementations. Examples off the top of my head are:
1/ Light threads with default implementation of a scheduler provided by CPython and not dissimilar to Golang. Orthogonal to adjunct libraries or potential language features like channels.
2/ A no-op that uses the GIL can be default or even an interpreter switch, as discussed in the article.
3/ A stable ABI so one can replace the CPython implementation with an alternative. Written in Rust or whatever could be for hardware or a cloud vendor's environment.
4/Sub-interpreters, where parallelism pays to be far more coarse, could be Actor like architectures where spawned processes are new interpreters whose Actor System is a graph of interpreters.
These envisioned use cases offer significant advantages, and the potential benefits are exciting and encouraging. The prospect of this progress is exhilarating, and I eagerly anticipate this endeavour's evolution. Thanks to all those painstakingly putting so much energy into this.
Edit: spelling and formatting
If single threaded performance is degraded, couldn't these people use an old version of python?
With such incredible increases in multithreaded performance, I imagine this is basically infrastructure tier importance. Like the US government should be funding it. Would throwing a billion dollars at it, solve it in ~1 year? Or is this going to take 3 years regardless?
What I hope the solution will end up emerging would be subinterpreters that are more intuitive to use. Honestly, I don't want to pass in source code, but rather a function or module entrypoint and have the subinterpreter abstract way the instantiation boilerplate.
At least I hope everyone is keeping an open mind and is willing (and was prepared) to backtrack if something doesn't seem to be working.
Ie. The programmer writes:
for file in glob.glob('*.jpg'):
data.append(load_file(file))
Rationale: Python is designed to be easy and simple. Concurrency in general is not that. Python should do only do concurrency in a way that can be made easy and simple.
* glob is an i/o operation: impure
* load_file is an i/o operation: impure
* data.append serializes part of the execution order
---
Interestingly, GPT4 does pretty ok:
files = glob.glob('*.jpg')
with ThreadPoolExecutor() as executor:
data = list(executor.map(load_file, files))
Separately, if just going for concurrency for i/o, async/await is pretty amazing:
tasks = [load_file(file) for file in glob.glob('*.jpg')]
data = await asyncio.gather(*tasks)
It sure was! I believe any automatic parallelization should be able to deal with all these things.
With a combination of opportunistic execution (ie. guess which bits can be parallelized, and roll back if wrong), and clever heuristics to avoid rollbacks, it should be pretty do-able.
CPU manufacturers do similar stuff at a smaller scale with parallel execution of single threaded code with Tomasulo's algorithm, and they get massive gains compared to non-superscalar CPU's. They even do similar stuff at medium scale with Hardware Lock Eliding.
That's not async or concurrent. You're running a synchronous function in an asynchronous task which cancels out.
> Rationale: Python is designed to be easy and simple. Concurrency in general is not that.
Python is way too dynamic for that. Furthermore, you have a syscall at the start of the loop, and one syscall per loop. Even if you did this in Rust, it probably would not give you the true results you wanted, if that file tree were manipulated in any sort of way during the loop, was a network FS, or any of a number of other assumptions were violated.
That's exactly why automatic concurrency is a bad idea. Now you have no idea if your code is executing in "simple mode" or "complex mode".
> Even if you did this in Rust, it probably would not give you the true results you wanted, if that file tree were manipulated in any sort of way during the loop, was a network FS, or any of a number of other assumptions were violated.
That's factually true but not really relevant.
Introducing your own parallelism barely adds to this problem. Your program cannot assume it is not parallel with another program affecting the same file tree in arbitrary ways.
The parallelism you add to your own code (say, with Rayon) doesn't change the set of operations that can occur to those files. If you design your own operations to be non-overlapping then parallelism is no concern, and if you didn't then that's a bug and you might get non-deterministic results even without parallelism.
For what it's worth, even for writing that kind of code, Rust's standard library has taken the recent wave of TOCTOU vulnerabilities much more seriously than contemporaries like C++ implementations. Good writeup here: https://www.reddit.com/r/cpp/comments/151cnlc/a_safety_cultu...
This of course only makes safer operations possible in more cases. Code still has to do what it can with the constraint that filesystem contents may change in arbitrary ways at arbitrary times. Adding your own parallelism does not add significantly to that. And if one is to add parallelism, it may as well be with something as high-level as Rayon and with all of the usual data race protection Rust provides, so that you do get to focus on filesystem concerns and not also on thread-safety in general.
However, consider parallel data processing, a task I face so often in computer vision. In C++, I can slap a `pragma omp parallel for` onto a for loop processing a list of images/videos/meshes/csvs, increasing the speed of this section of the program by the number of CPU cores. I do this when the problem is well defined (one input, one output each), and it hasn't blown up in my face once. In Python, on the other hand, it feels cumbersome when spinning up worker processes for such tasks, not to mention harder to debug. For this kind of work, removing the GIL would be a massive relief.
It's good that people are highlighting the issues. But to add my weight to the scale, the GIL is the main barrier in Python to stand in my way to speed up my programs. So I'm very happy that they're investing their resources to build a proper way of doing thread-parallelism.
I suspect the negativity you're seeing comes mainly from two main sources. First people are terrified of another 2->3 situation. If removing the GIL ends up breaking any existing libraries or causing significant single-core performance degradations then that could do seriously damage to python and no one wants to go through that again.
Secondly people who have been working on this for a long time get annoyed when people make removing the GIL sound easy and imply that the only reason it hasn't been done is because the developers are lazy, incompetent or haven't thought of it. Various people have been trying for literally decades, but so far no one has managed to come up with a solution that is both backwards compatible and performant.
Everybody agrees that removing the GIL would be fantastic. However most of the core developers aren't willing to sacrifice backwards compatibility with existing code to get there.
Even if the GIL is removed, the task you’re talking about would be dozens of times slower in Python than in a better-suited language, so why not write it in C++ or Rust and then call into that from Python?
I could be wrong or missing something, and I’m not a Python developer so I don’t really have a dog in the fight, but I just wanted to try to explain where some of the criticism is coming from.
I warmly recommend reading the motivation section of the PEP. It is incredibly well written, quoting issues from PyTorch, scikit and NumPy:
What am I missing?
Again, I would encourage you to read the motivation section of the PEP, which shows a multitude of frustrations with the current state. The issue is subtle enough that I'm struggling to summarize it in a single sentence.
Computer vision often has tasks independent of the global state; pure functions, if you will. But many applications have different "state" pools all over the place. JIRA is written in Python. Their library for accessing their API alone is 8k lines of Python. I suspect JIRA's code base won't be all pure functions.
A large group (probably a comfortable majority) uses Python at a basic level. They may use numpy and matplot, but that's about it. They will notice a slow down when the GIL is removed. Don't expect smiles.
More advanced users may use concurrent code. If you use an external C library, chances are it has to be updated to work with a no-gil python version. This won't go swimmingly. It puts the burden on the maintainers of those libs, and they may not be able or willing to fix the problem. That will lead to unhappy faces.
And then there will be the subtle bugs in complex systems. Python programmers are not really used to synchronicity problems, so they'll slap locks all over the place, leading to slow down or dead-locks. More unhappy faces.
So I see the commitment to a single, no-GIL version as a serious threat to Python.
That's what I suspect happening on the Python / scientific-computing community. They thought (and some people in this thread still think) that Python can be the driver to wait for compute-bounded task to finish (effectively, making the Python code to be IO-bounded / i.e. waiting for another language to finish). But over time, it changed. People write more code in the Python driver, and suddenly, the very optimized C code is no longer the bottleneck, the Python code is.
We've seen this bouncing pattern enough times and really need to think through whether we want to have two different design patterns to deal with IO-bounded problem v.s. compute-bounded problem.
There was a question from Shannon about ""what people think is a
acceptable slowdown for single-threaded code"" ... he had estimated
an impact "in the 15-20% range"
To me, the acceptable slowdown is exactly zero.
I can already use multiple cores by running multiple Python processes.
Any slowdown of single-process performance would be a terrible step backwards.
Python is already slow. It should look at PHP and see what it can take out of their book. PHP 7, which had no JIT, was already about 6x faster than Python. PHP 8 is even faster for some workloads. I'm not sure if the overhead of a JIT makes sense though. But PHP 7 is a good place to look for a performance benchmark.
This has been the reason most of the previous GIL removal projects failed. GvR had a strict zero slowdown in single-thread performance rule for any nogil patches and none of them managed that. I wonder if the core team has decided to abandon that rule now that GvR is gone?
Because you have terribly simple processes that don't need to do synchronization. Even a simple parallel map would have Python scream and kick, because the GIL prevents it from having any kind of reasonable performance, and your multiprocess can't handle that unless they all write to files.
If you care about single % of performance, why are you using Python in the first place? You're likely losing 100x to more performant languages already.
PHP got threads late and as an extension. The issues with the GIL centre around the C extension API. PHP doesn't have to deal with that, so they were able to concentrate on making single threaded performance as good as possible. The GIL also blocks some potential avenues around making single threaded performance better, which is why the plan for this current attempt is to make it opt-in.
Also keep in mind that all this is happening concurrently with efforts around making single threaded performance better, with the idea bring that any loss of performance due to the GIL removal should be compensated by other speed improvements.
If we could go back in time and make the C API more like that of, say, Lua, that would solve the issue, but we can't.
Where does that number come from? My current desktop has 32 logical cores and it has been a few years since I bought that one.
If you can parallelise 90% of your code, you get only 10x improvement even on infinite number of cores
My 64 core server setup says otherwise.
> Is it really worth it?
Yes, very much so.
> If this makes it even slightly more difficult to develop Python for normal people
It won't. Pure python code will very likely not notice a difference.
A very characteristic thing for a core Python developer to say: it's stupid, baseless, but gets a standing ovation.
The reason Python 3 wasn't catching up is because it offered no tangible benefits. Had it been, say, 10x faster than Python 2, then there would've been a reason. Had it offered sensible concurrency, there would've been a reason, had it offered some way to automatically verify programs... or a bunch of other useful things you might expect from a language, there would've been a reason to switch. But Python 3 was and still is a worthless change. It just gives you the ability to do the same things you could already do but in a slightly different way. If CPython project didn't refuse to support Python 2, it would be still alive and well.
I've lived for a while with the language that had to support two versions at the same time: ActionScript. After AS3 was released, the SWF format and Flash player had changed to support executing both at the same time. The cases when both were used at the same time were very rare. They were never a part of an upgrade process -- usually it was because someone had to embed code they were under contractual obligation to execute (s.a. showing ads). Nobody partially upgraded their projects.
Now, even though Python didn't provide a way to run two versions of the language at the same time, it wouldn't have been hard to do on your own, if you really wanted that: say modify the multiprocessing module to launch Python 2 from Python 3 or Python 3 from Python 2. Or create a native module embedding a Python interpreter of a different version. There are other ways as well... but nobody was doing that. Nobody needed that.
There's no evidence anyone wanted to run both versions of Python at the same time, but, instead of facing the music and understanding that Python 3 wasn't wanted, these people will pet themselves on the back, tell each other how great their project is, and how it doesn't succeed for any other reason beside them being incompetent.
And when there's a thread about Go I'm sure someone will tell us they don't use it and a different language is 100x better, too.