PEP 684 was accepted – Per-interpreter GIL in Python 3.12 | Hacker News Reader

PEP 684 was accepted – Per-interpreter GIL in Python 3.12

twitter.com

75 pointsbratao13 comments

This is great and was a huge effort by Eric Snow and collaborators to make it happen.

To be clear, this is not "PEP 703 – Making the Global Interpreter Lock Optional in CPython", by Sam Gross. PEP 703 is more ambitious and add quite a bit of complexity to the CPython implementation. PEP 684 mostly cleans things up and encapsulates global state better. The most distruptive change is to allow immortal objects. I hope PEP 703 can go in too, even with the downsides (complexity, bit slower single threaded performance).

"PEP 703 – Making the Global Interpreter Lock Optional in CPython" (2023) https://github.com/python/peps/blob/main/pep-0703.rst https://peps.python.org/pep-0703/

colesbury/nogil https://github.com/colesbury/nogil :

  docker run -it nogil/python
  docker run -it nogil/python-cuda

Are the immortal objects different than what happens with gc.freeze() ?

Moderately different; Particularly with respect to how to performance.

If I read this correctly immortal objects stop changing refcount which prevents cache line invalidation while `gc.freeze()` just stops cleaning up objects with zero refcount.

Oh, that would be amazing for quite a few things in web apps if exposed properly. Love it.

So after this and some follow-on work, you can switch from multiprocess message passing parallelism to same-process message passing multi-interpreter parallelism? Which would decrease message passing overhead at cost of crash/memory safety isolation.

Yes, that what I´m excited about. A ThreadPoolExecutor like implementation that effectively use multiple CPUs.

ThreadPoolExecutor seems to be a concept from Java land which is pretty different, you can share objects between Java threads.

Replying to myself, there's also a identically named class in Python nowadays. But the point remains, you won't get that type of shared-memory parallel multithreading from this. (Which is good I think as I'm a fan of Erlang style message passing parallelism)

This is really cool and I'm excited about learning this new concurrency model, which isn't something I've ever said before or expected to say.

Most languages I used, python included, haven't done concurrency in a way that added enough without taking so much away where I have been inclined to use it by default. Maybe this is a change from that for python.

Nudging closer to the Beam, eh?

Now we just need PEP 554 so this can actually be used from Python code.

https://peps.python.org/pep-0554/ "Multiple Interpreters in the Stdlib"

Reply on news.ycombinator.com