Hacker News Reader

Comment by simonw | Hacker News Reader

https://github.com/faster-cpython/ideas/wiki/Python-3.12-Goa... is interesting too.

> Python currently has a single global interpreter lock per process, which prevents multi-threaded parallelism. This work, described in PEP 684, is to make all global state thread safe and move to a global interpreter lock (GIL) per sub-interpreter. Additionally, PEP 554 will make it possible to create subinterpreters from Python (currently a C API-only feature), opening up true multi-threaded parallelism.

Very basic question: in a world where a Python program can spin up multiple subinterpreters, each of which can then execute on a separate CPU core (since they don't share a GIL), what will the best mechanisms be for passing data between those subinterpreters?

View on HN

> There are a number of valid solutions, several of which may be appropriate to support in Python. This proposal provides a single basic solution: “channels”. Ultimately, any other solution will look similar to the proposed one, which will set the precedent. Note that the implementation of Interpreter.run() will be done in a way that allows for multiple solutions to coexist, but doing so is not technically a part of the proposal here.

> Regarding the proposed solution, “channels”, it is a basic, opt-in data sharing mechanism that draws inspiration from pipes, queues, and CSP’s channels.

> As simply described earlier by the API summary, channels have two operations: send and receive. A key characteristic of those operations is that channels transmit data derived from Python objects rather than the objects themselves. When objects are sent, their data is extracted. When the “object” is received in the other interpreter, the data is converted back into an object owned by that interpreter.

https://peps.python.org/pep-0554/#shared-data

> Along those same lines, we will initially restrict the types that may be passed through channels to the following:

> * None

> * bytes

> * str

> * int

> * channels

> Limiting the initial shareable types is a practical matter, reducing the potential complexity of the initial implementation.

That's a really interesting detail - presumably channels can be passed so you can do callbacks ("reply on this other channel").

I wonder why floats aren't on that list? I know they're more complex than ints, but I would expect they would still end up with a relatively simple binary representation.

I don’t know why floats aren’t included, but any float can be easily represented by an int with same bits, or a bytestring, using the struct module to convert between them, so there are clear workarounds.

Sounds like an oversight…

Passing channels over other channels used to be a reasonably standard trick in Go, IIRC.

Fun fact, floats are easier than `int` because `float` has a predictable size.

> That's a really interesting detail - presumably channels can be passed so you can do callbacks ("reply on this other channel").

Callbacks or notifications, yes. I use both patterns quite often.

As someone who uses channels all the time (in Nim) for cross-thread comms, this is pretty exciting. The deep-copy that Nim channels do makes things simpler at the cost of more memory allocations obviously, but even on an ESP32-S3 it's been a great abstraction. Of course I get to cheat and use actual shared memory with FreeRTOS semaphores/mutexes and such when its really required, but having channels as the first-class easy-to-use mechanism is the right move in my opinion (which is worth about as much as you just paid for it, of course)

Any idea what a "channel" actually is/how it is implemented?

Wild guess, a double-ended queue guarded by some kind of locking mechanism ?

> Very basic question: [not basic at all question which has been the subject of decades of research and produced several specialized programming models]

(Brackets my own of course.)

Sharing data in concurrent programs is not trivial, especially in environments where data is mutable. The most trivial answer to the question is “message passing”, as in the SmallTalk notion of OOP or the Erlang/OTP Actor Model. Some solutions look much more like working with a database (Software Transactional Memory). Some models that seem entirely designed for a different problem space are also compelling (various state models common in UI and games like reactivity and Entity Component systems).

PyPy pursued the STM route many years ago but later abandoned it.

https://doc.pypy.org/en/latest/stm.html

Is there an explanation of why they abandoned it ? It seemed very cool at that time...

I was gonna ask why it was abandoned but then I read through the caveats, and well… yikes!

This plan sounds very much like Ruby Ractors, which are essentially sub-interpreters, each with their own GVL.

Shareable data is basically immutable data + classes/modules, and unshareable data can be transmitted via push (send+receive) or pull (yield+take). Transmission implies either deep copying (which "forks" the instances) or moving with ownership change (sender then loses access)

See here for details: https://docs.ruby-lang.org/en/master/ractor_md.html

Depends on what criteria you use for "best".

If it's performance, then, since subinterpreters run in the same process, it would be global shared state. You can't use Python objects across subinterpreters, but raw byte arrays will work just fine, provided you do your own locking correctly around all that.

Python would need to implement a multiconsumer multiproducer ringbuffer or a blocking free algorithm (I'm not sure if it is wait free) such as the actor system I implemented below

To apply to Python The subinterpreters could transfer ownership of the refcounts between subinterpreters as part of an enqueue and dequeue.

I believe the refcount locking approach has scalability problems between threads.

I implemented a multithreaded actor system with work stealing in Java and message passing can get to throughputs of around 50-60 million messages per second without blocking or mutexes. The only lock is not quite a spinlock. I use an algorithm I created but inspired by this whitepaper [1], which is simple but works. It's probably a known algorithm but I'm not sure of the name of it.

I have a multidimensional array of actor inboxes (each actor has multiple buffers for filling by other threads to lower contention to 0) then there is an integer stored for the thread that is trying to read or write to the critical section.

The threads all scan this multidimensional array forwardly and backwardly to see if another thread is in the critical section. If nobody is there, it marks the critical section. It then scans again to see if it is still valid. it's similar to going into a room and scanning the room left and scanning the room right. Surprisingly this leads to thread safely. I wrote a python model checker to verify the algorithm is correct.

Without message generation within threads, it can communicate and sum 1 billion integers in 1 second due to parallelism (it takes 2 seconds to do this with one thread) It takes advantage of the idea that variable assignment can transfer any amount of data in an assignment.

See Actor2.java (1 billion sums a second messages created in advance), Actor2MessageGeneration.java (20 million requests per second, messages created as we go) or Actor2ParallelMessageCreation.java (50-60 million requests per second, with parallel message creation)

There's also a Java multiconsumer multiproducer ringbuffer in this repository [3] too which I ported from Alexander Krizhanovsky [2]

[1]: https://lag.net/papers/content/leftright-extended.pdf

[2]: https://www.linuxjournal.com/content/lock-free-multi-produce...

[3]: https://github.com/samsquire/multiversion-concurrency-contro...

Redis or some handroller message queue

> move to a global interpreter lock (GIL) per sub-interpreter

hallelujah!

Global variables are evil. The fact that sub-interpreters aren’t currently possible in Python is one of my canonical examples of why they’re evil.

Reply on news.ycombinator.com