Depending on the domain, the reality can be the reverse.
Multiprocessing in the web serving domain, as in "spawning separate processes", is actually simpler and less bug-prone, because there is considerably less resource sharing. The considerably higher difficulty of writing, testing and debugging parallel code is evident to anybody who's worked on it.
As for the overhead, this again depends on the domain. It's hard to quantify, but generalizing to "massive" is not accurate, especially for app servers with COW support.
The default for multiprocessing is still to fork (fortunately changing in 3.14), which means all of your parent process’ threaded code (incl. third party libraries) has to be fork-safe. There’s no static analysis checks for this.
This kind of easy to use but incredibly hard to use safely library has made python for long running production services incredibly painful in my experience.
[1] Some arguments to subprocess.popen look handy but actually cause python interpreter code to be executed after the fork and before the execve, which has caused production logging-related deadlocks for me. The original author was very bright but didn’t notice the footgun.
If I may: Changing from fork to what?
As an aside I still constantly see side effects in imports in a ton of libraries (up to and including resource allocations).
Compared to theads being "pain free"?
Abruptly terminating a child process still can potentially cause issues, but there are whole categories of potential issues which exist for abrupt thread termination but not for abrupt process termination.
If you have an open file or network connection, the kernel is guaranteed to close it for you when the process is killed (assuming it hasn't passed the fd/socket to a subprocess, etc). That's not a matter of opinion.
Yes, if you are writing to a file, it is possible abruptly killing the writer may leave the file in an inconsistent state. But maybe you know your process doesn't write to any files (that's true in my case). Or maybe it does write to files, but you already have other mechanisms to recover their integrity in this scenario (since file writing processes can potentially die at any time–kernel panic, power loss, intermittent crash bug, etc)
That's not useful feedback.
The only issue there is that sometimes library code will incorrectly defer the exception (i.e. suppress it) but otherwise it's pretty good.
I feel like all of this is tragic and Python should have gone to a BEAM-like model some years ago, like as part of the 2 to 3 transition. Instead we get async wreckage and now free threading with its attendant hazards. Plus who knows how many C modules won't be expecting this.
https://www.youtube.com/watch?v=bzkRVzciAZg
This is also good:
https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
web search on "colored functions" finds lots of commentary on that article.
In short, "await" gives me an extra piece of data about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing.
There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them.
EDIT: all of the above is predicated on the idea that reading code is harder than writing it. If you believe the opposite, then blue/red has a point.
To me the difficulty is more with writing generic code and maintaining abstraction boundaries. Unless the language provides a way to generalise over asyncness of functions, we need a combinatorial explosion of async variants of generic functions. Consider a simple filter algorithm it needs versions for: (synchronous vs asynchronous iterator) times (synchronous vs asynchronous predicate). We end up with a pragmatic but ugly solution: provide 2 versions of each algorithm: an async and a sync, and force the user of the async one to wrap their synchronous arguments.
Similarly changing some implementation detail of a function might change it from a synchronous to an asynchronous function, and this change must now propagate through the entire call chain (or the function must start its own async runtime). Again we end up in a place where the most future proof promise to give for an abstraction barrier is to mark everything as async.
This, for me, is the main drawback of async/await, at least as it is implemented in for example Python. When you call a synchronous function which makes network calls, then it blocks the event loop, which is pretty disastrous, since for the duration of that call you lose all concurrency. And it's a fairly easy footgun to set off.
> It is a rather vague signal about the functions behavior as opposed to the lack of the IO monad in Haskell.
I'm happy you mentioned the IO monad! For me, in the languages people pay me to write in (which sadly does not include Haskell or F#), async/await functions as a poor man's IO monad.
> Again we end up in a place where the most future proof promise to give for an abstraction barrier is to mark everything as async.
Yes, this is one way to write async code. But to me this smells the same as writing every Haskell program as a giant do statement because the internals might want to do I/O at some point. Async/await makes changing side-effect free internals to effectful ones painful, which pushes you in the direction of doing the I/O at the boundaries of your system (where it belongs), rather than all over the place in your call stack. In a ports-adapters architecture, it's perfectly feasible to restrict network I/O to your service layer, and leave your domain entirely synchronous. E.g. sth like
async def my_service_thing(request, database):
my_business_object = await database.get(request.widget_id)
my_business_object.change_state(request.new_widget_color) # some complicated, entirely synchronous computation
await database.save(my_business_object)
Async/await pushes you to code in a certain way that I believe makes a codebase more maintainable, in a way similar to the IO monad. And as with the IO monad, you can subvert this push by making everything async (or writing everything in a do statement), but there's better ways of working with them, and judging them based on this subversion is not entirely fair.> ugly solution: provide 2 versions of each algorithm: an async and a sync
I see your point, and I think it's entirely valid. But having worked in a couple async codebases for a couple of years, the amount of stuff I (or one of my collaborators) have had to duplicate for this reason I think I can count on one hand. It seems that in practice this cost is a fairly low one.
Do you think we should be annotating functions with `expensive` and/or `networking`? And also annotating all of their callers, recursively? And maintaining 4 copies of every higher-order function depending on whether the functions it calls are `expensive`, `networking`, neither or both?
No, we rely on documentation for those things, and IMO we should for `async` as well. The reason we can’t, and why `async`/`await` exist, is because of shortcomings (lack of support for stackful coroutines) in language runtimes. The best solution is to fix those shortcomings, not add viral annotations everywhere.
For example, I much prefer a signature like
def f(a: int) -> str:
over def f(a):
because it allows me to see, without reading the implementation of the function (or, if it exists, and I'm willing to bet on its reliability, the documentation), that it takes an integer, and gives me a string. And yes, this requires that I write viral type annotations on all my functions when I write them, but for me the bottleneck at my job is not writing the code, it's reading it. So that's a small upfront cost I'm very much willing to pay.> Do you think we should be annotating functions with `expensive` and/or `networking`? And also annotating all of their callers, recursively?
Yes, absolutely, and yes, absolutely. That's just being upfront and honest about an intrinsic property of those functions. A function calling a function that does network I/O by transitivity also does network I/O. I prefer code that's explicit over code that's implicit.
Fair enough, that's a valid philosophy, and one in which `async`/`await` makes perfect sense.
However, it's not Python's philosophy - a language with dynamic types, unchecked exceptions, and racy multithreading. In Python, `async`/`await` seems to be at odds with other language features - it feels like it's more at home in a language like Rust.
python3 -c "import this" | head -4 | tail -1
I think the fast and loose style that Python enables is perfect for small scripts and one off data science notebooks and the like. But having worked in large codebases which adopt the same style, and ones that avoid it through static typing and in some cases async/await, the difference in productivity I've noticed in both me and my collaborators is too stark for me to ignore.I think I should've been more nuanced in my comments praising async/await. I believe that what I say is valid in large IO-bound applications which go beyond basic CRUD operations. In general it depends, of course.
Agreed - I only use Python for scripts like this, preferring statically-typed, AOT-compiled languages for larger programs.
That’s why I think Python should have adopted full coroutines - it should play to its strengths and stick to its fast-and-loose style. However, the people who decide how the language evolves are all employees of large companies using it for large codebases - their needs are very different from people who are only using Python for small scripts.
The JVM runtime has solved this problem neatly with virtual threads in my opinion. Run a web request in a virtual thread, and all blocking I/O is suddenly no longer blocking the OS thread, but yielding/suspending and giving and giving another virtual thread run time. And all that without language keywords that go viral through your program.
Lots of microcontroller OS's use cooperative multitasking but once there are enough machine resource's, OS's generally become preemptive. Async concurrency is basically cooperative multitasking with similar issues. Does Python give a way to open a file asynchronously in Linux? It's now possible with io_uring but it was impossible for a very long time (like decades). Erlang and GHC both use thread pools to deal with that. The use the old synchronous open(2) call but move it into an auxiliary thread so it won't block the calling thread.
That doesn't sound like real hatred. Those sound like real concerns, which need to be addressed, and the attempt to remove the GIL is doing so very much with those concerns in mind.
On the other hand async functions can be very cheap.
Again, which useful property does async protect?
Also, I've found that ChatGPT/Claude3.5 are much, much smarter and better at Python than they are at C++ or Rust. I can usually get code that works basically the first or second time with Python, but very rarely can do that using those more performant languages. That's increasingly a huge concern for me as I use these AI tools to speed up my own development efforts very dramatically. Computers are so fast already anyway that the ceiling for optimization of network oriented software that can be done in a mostly async way in Python is already pretty compelling, so then it just comes back again to developer productivity, at least for my purposes.
It will be interesting to see how this goes over the next few years. My guess is that a lot of lessons were learned from the python 2 to 3 move. This plan seems pretty solid.
And of course there's a relatively easy fix for code that can't work without a GIL: just do what people are doing today and just don't fork any threads in python. It's kind of pointless in any case with the GIL in place so not a lot of code actually depends on threads in python.
Preventing the forking of threads in the presence of things still requiring the GIL sounds like a good plan. This is a bit of meta data that you could build into packages. This plan is actually proposing keeping track of what packages work without a GIL. So, that should keep people safe enough if dependency tools are updated to make use of this meta data and actively stop people from adding thread unsafe packages when threading is used.
So, I have good hopes that this is going to be a much smoother transition than python 2 to 3. The initial phase is probably going to flush out a lot of packages that need fixing. But once those fixes start coming in, it's probably going to be straightforward to move forward.
AMD EPYC 9754 with 128-cores/256-threads, and EPYC 9734 with 112-cores/224-threads. TomsHardware says they "will compete with Intel's 144-core Sierra Forest chips, which mark the debut of Intel's Efficiency cores (E-cores) in its Xeon data center lineup, and Ampre's 192-core AmpereOne processors".
What in 5 years? 10? 20? How long will "1 core should be enough for anyone using Python" stand?
There is code that may benefit from the free threaded implementation but it is not as often as it might appear and it is not without its own downsides. In general, GIL simplifies multithreaded code.
There were no-GIL Python implementations such as Jython, IronPython. They hadn't replaced CPython, Pypy implementation which use GIL i.e., other concerns dominate.
If you're looking for a 32x or 128x performance improvement from python supporting multi-core you should probably rewrite in C, C++, Rust, or Fortran and get that 100x improvement today on a single core. If done properly you can then ALSO get the gain from multiple cores on top of that. Or to put it another way, if performance is critical python is a poor choice.
> "Or to put it another way, if performance is critical python is a poor choice."
To put it another way, just because performance isn't critical, doesn't mean that more performance for free is not desirable or beneficial, or that ignoring 127/128ths of available performance is fine.
A piece of code takes 6h to develop in C++, and 1h to run.
The same algorithm takes 3h to code in Python, but 6h to run.
If I could thread-spam that Python code on my 24 core machine, going Python would make sense. I've certainly been in such situations a few times.
Julia is one that is gaining a lot of use in academia, but any number of modern, garbage collected compiled high level languages could probably do.
(And as a side note: I have never, in around a decade of writing C++, heard std::thread described as "easy to debug.")
I expected that dropping down to C/C++ would be a large jump in difficulty and quantity of code, but I've found it isn't, and the dev experience isn't entirely worse, as, for example, in-editor code-intelligence is rock solid and very fast in every corner of my code and the libraries I'm using.
If anyone could benefit from speeding up some Python code, I'd highly recommend installing cppyy and giving it a try.
To be honest, I don't know much about the speed, as my use-case isn't speeding up slow code.
Every DL library comes with its own C++ backend that does this for now, but it's annoyingly inflexible. And dealing with GIL is a nightmare if you're dealing with mixed Python code.
in Python language specifically. Their library may have already done some form of parallelization under the hood
There are a lot of simple cases where multi-threading can easily triple or quadruple the performance.
I used to write a ton of MPI based parallel python. It's pretty straightforward. But one could easily imagine trying to improve the multiprocessing ergonomics rather than introducing threading. Obviously the people who made the choice to push forward with this are aware of these options, too. Still mildly puzzling to me why threads for Python are needed/reasonable.
Since Python leads over any other languages in the ML community, and ML is a hot topic right now, it makes sense for Python developers to secure the lead by making the life of ML developers easier, which is by introducing GIL-less multi-threading.
But the mai point against multiprocessing seems to be that spawning new processes is slow ...
That single "alternatives" paragraph doesn't answer at all why mp isn't viable for python level parallelism.
I am no longer invested in python heavily, I am sure there are discussions or documents somewhere that go into this more. Might be that it's simply that everyone is used to threads so you should support it for sheer familiarity. All I am saying is it's not obvious to a casual observer.
IDK what l should and shouldn't be written in, but there are a very large # of proud "pure Python" libraries on GitHub and HN.
The ecosystem seems to even prefer them.
Why shouldn't someone who prefers writing in python benefit from using multiple cores?
I did use the words "most things". I'm not saying this is a bad development for Python, or that nobody should use it. But if performance is a top priority, Python is the wrong language and always has been.
I use Python from time to time, it's fun and easy to put certain kinds of things together quickly. But each time I do a project with it, the first thing I ask myself is "is this going to be fast enough?" If not I'll use something else.
This just isn’t true.
This does not improve single threaded performance (it’s worse) and concurrent programming is already available.
This will make it less annoying to do concurrent processing.
It also makes everything slower (arguable where that ends up, currently significantly slower) overall.
This way over hyped.
At the end of the day this will be a change that (most likely) makes the existing workloads for everyone slightly slower and makes the lives of a few people a bit easier when they implement natively parallel processing like ML easier and better.
It’s an incremental win for the ML community, and a meaningless/slight loss for everyone else.
At the cost of a great. Deal. Of. Effort.
If you’re excited about it because of the hype and don’t really understand it, probably calm down.
Mostly likely, at the end of the day, it s a change that is totally meaningless to you, won’t really affect you other than making some libraries you use a bit faster, and others a bit slower.
Overall, your standard web application will run a bit slower as a result of it. You probably won’t notice.
Your data stack will run a bit faster. That’s nice.
That’s it.
Over hyped. 100%.
The rest of us can live with arcane threading bugs and yet another split ecosystem. As I understand it, if a single C-extension opts for the GIL, the GIL will be enabled.
Of course the invitation to experiment is meaningless. CPython is run by corporations, many excellent developers have left and people will not have any influence on the outcome.
Man, that phrase perfectly encapsulates so much of Python’s evolution over the last ~10 years.
If you assume two completely separate implementations where there is an #ifdef every 10 lines and atomics and locking only occur with --disable-gil, there is no slowdown for the --enable-gil build.
I don't think that is entirely the case though!
If the --enable-gil build becomes the default in the future, then peer pressure and packaging discipline will force everyone to use it. Then you have the OBVIOUS slowdown of atomics and of locking the reference counting and in other places.
The advertised figures were around 20%, which would be offset by minor speedups in other areas. But if you compare against Python 3.8, for instance, the slowdowns are still there (i.e., not offset by anything). Further down on the second page of this discussion numbers of 30-40% have been measured by the submitter of this blog post.
Actual benchmarks of Python tend to be suppressed or downvoted, so they are not on the first page. The Java HotSpot VM had a similar policy that forbid benchmarks.
^ read. The OP responds in the thread.
tldr, literally what I said:
> It also makes everything slower (arguable where that ends up, currently significantly slower) overall.
longer version:
If there was no reason for it to be slower, it would not be slower.
...but, implementing this stuff is hard.
Doing a zero cost implementation is really hard.
It is slower.
Where it ends up eventually is still a 'hm... we'll see'.
To be fair, they didn't lead the article here with:
> Right now there is a significant single-threaded performance cost. Somewhere from 30-50%.
They should have, because now people have a misguided idea of what this wip release is... and that's not ideal; because if you install it, you'll find its slow as balls; and that's not really the message they were trying to put out with this release. This release was about being technically correct.
...but, it is slow as balls right now, and I'm not making that shit up. Try it yourself.
/shrug
But, for sure, nogil will be good for those workloads written in pure Python (though I've personally never been affected by that).
I use coroutines and multiprocessing all the time, and saturate every core and all the IO, as needed. I use numpy, pandas, xarray, pytorch, etc.
How did this terrible GIL overhead completely went unnoticed?
That means your code is using python as glue and you do most of your work completely outside of cPython. That's why you don't see the impact - those libraries drop GIL when you use them, so there's much less overhead.
I've never heard threading described as "simple", even less so as simpler than multiprocessing.
Threads means synchronization issues, shared memory, locking, and other complexities.
Everyone wants parallelism in Python. Removing the GIL isn't the only way to get it.
I'm saturating 192cpu / 1.5TBram machines with no headache and straightforward multiprocessing. I really don't see what multithreading will bring more.
What are these massive overheads / complexity / bugs you're talking about ?