Hacker News Reader

Comment by joelthelion | Hacker News Reader

Perfomance,I suppose? Spawning a process is expensive.

I understood that on Linux using a process was similar in speed to a thread, though they're much slower on Windows. Has that changed?

For my python multi core code I like using fork with sending objects over a socket with pickle. It gives more control than multiprocessing.

It works pretty well. One downside of fork with python is that the reference counters are scattered about, leading to big COW memory churn, however. Big numpy arrays should be shared, however.

Spwaning the process is not the only problem, communicating between them is also expensive.

This makes sense. Combine it with asyncio and you have a full Node-like runtime without relying on Uvicorn and the like (even though clustering is still good for scaling).

If you're using asyncio, I don't see what having a subinterpreter gains you. Asyncio by itself is already a Node-like runtime without relying on third-party libraries.

True true, my comment was not clear. With asyncio it's still a single threaded event loop (that can be clustered like Node or any Python processes as has always been done).

With sub-interpreters, I wonder how different it is from normal multithreading like in JVM. Need to read up on this idea. I can't see how it wouldn't require the same locking and mutexes or message passing if multiple interpreters are to work on logically related data.

So my immediate thoughts were about leveraging it as a replacement for multithreading and event loop clustering by treating the interpreters as lightweight processes and having them communicate over some kind of protocol. Like how the BEAM does built-in process supervision.

The "supervisor" process itself is async in the way it coordinates the tree.

Aaand I finally read the article and PEP and there's no escape from the GIL :( Damn wishful thinking.

Reply on news.ycombinator.com