I feel like most things that will benefit from moving to multiple cores for performance should probably not be written in Python. OTH "most" is not "all" so it's gonna be awesome for some.
I feel like all of this is tragic and Python should have gone to a BEAM-like model some years ago, like as part of the 2 to 3 transition. Instead we get async wreckage and now free threading with its attendant hazards. Plus who knows how many C modules won't be expecting this.
https://www.youtube.com/watch?v=bzkRVzciAZg
This is also good:
https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...
web search on "colored functions" finds lots of commentary on that article.
In short, "await" gives me an extra piece of data about the function, without having to read the body of the function (and the ones it calls, and the ones they call, etc). That's a good thing.
There are serious drawbacks to async/await, and the red/blue blog post manages to list none of them.
EDIT: all of the above is predicated on the idea that reading code is harder than writing it. If you believe the opposite, then blue/red has a point.
To me the difficulty is more with writing generic code and maintaining abstraction boundaries. Unless the language provides a way to generalise over asyncness of functions, we need a combinatorial explosion of async variants of generic functions. Consider a simple filter algorithm it needs versions for: (synchronous vs asynchronous iterator) times (synchronous vs asynchronous predicate). We end up with a pragmatic but ugly solution: provide 2 versions of each algorithm: an async and a sync, and force the user of the async one to wrap their synchronous arguments.
Similarly changing some implementation detail of a function might change it from a synchronous to an asynchronous function, and this change must now propagate through the entire call chain (or the function must start its own async runtime). Again we end up in a place where the most future proof promise to give for an abstraction barrier is to mark everything as async.
This, for me, is the main drawback of async/await, at least as it is implemented in for example Python. When you call a synchronous function which makes network calls, then it blocks the event loop, which is pretty disastrous, since for the duration of that call you lose all concurrency. And it's a fairly easy footgun to set off.
> It is a rather vague signal about the functions behavior as opposed to the lack of the IO monad in Haskell.
I'm happy you mentioned the IO monad! For me, in the languages people pay me to write in (which sadly does not include Haskell or F#), async/await functions as a poor man's IO monad.
> Again we end up in a place where the most future proof promise to give for an abstraction barrier is to mark everything as async.
Yes, this is one way to write async code. But to me this smells the same as writing every Haskell program as a giant do statement because the internals might want to do I/O at some point. Async/await makes changing side-effect free internals to effectful ones painful, which pushes you in the direction of doing the I/O at the boundaries of your system (where it belongs), rather than all over the place in your call stack. In a ports-adapters architecture, it's perfectly feasible to restrict network I/O to your service layer, and leave your domain entirely synchronous. E.g. sth like
async def my_service_thing(request, database):
my_business_object = await database.get(request.widget_id)
my_business_object.change_state(request.new_widget_color) # some complicated, entirely synchronous computation
await database.save(my_business_object)
> ugly solution: provide 2 versions of each algorithm: an async and a sync
I see your point, and I think it's entirely valid. But having worked in a couple async codebases for a couple of years, the amount of stuff I (or one of my collaborators) have had to duplicate for this reason I think I can count on one hand. It seems that in practice this cost is a fairly low one.
Do you think we should be annotating functions with `expensive` and/or `networking`? And also annotating all of their callers, recursively? And maintaining 4 copies of every higher-order function depending on whether the functions it calls are `expensive`, `networking`, neither or both?
No, we rely on documentation for those things, and IMO we should for `async` as well. The reason we can’t, and why `async`/`await` exist, is because of shortcomings (lack of support for stackful coroutines) in language runtimes. The best solution is to fix those shortcomings, not add viral annotations everywhere.
The JVM runtime has solved this problem neatly with virtual threads in my opinion. Run a web request in a virtual thread, and all blocking I/O is suddenly no longer blocking the OS thread, but yielding/suspending and giving and giving another virtual thread run time. And all that without language keywords that go viral through your program.
Lots of microcontroller OS's use cooperative multitasking but once there are enough machine resource's, OS's generally become preemptive. Async concurrency is basically cooperative multitasking with similar issues. Does Python give a way to open a file asynchronously in Linux? It's now possible with io_uring but it was impossible for a very long time (like decades). Erlang and GHC both use thread pools to deal with that. The use the old synchronous open(2) call but move it into an auxiliary thread so it won't block the calling thread.
That doesn't sound like real hatred. Those sound like real concerns, which need to be addressed, and the attempt to remove the GIL is doing so very much with those concerns in mind.
For example, I much prefer a signature like
def f(a: int) -> str:
def f(a):
> Do you think we should be annotating functions with `expensive` and/or `networking`? And also annotating all of their callers, recursively?
Yes, absolutely, and yes, absolutely. That's just being upfront and honest about an intrinsic property of those functions. A function calling a function that does network I/O by transitivity also does network I/O. I prefer code that's explicit over code that's implicit.
Fair enough, that's a valid philosophy, and one in which `async`/`await` makes perfect sense.
However, it's not Python's philosophy - a language with dynamic types, unchecked exceptions, and racy multithreading. In Python, `async`/`await` seems to be at odds with other language features - it feels like it's more at home in a language like Rust.
python3 -c "import this" | head -4 | tail -1
I think I should've been more nuanced in my comments praising async/await. I believe that what I say is valid in large IO-bound applications which go beyond basic CRUD operations. In general it depends, of course.
Agreed - I only use Python for scripts like this, preferring statically-typed, AOT-compiled languages for larger programs.
That’s why I think Python should have adopted full coroutines - it should play to its strengths and stick to its fast-and-loose style. However, the people who decide how the language evolves are all employees of large companies using it for large codebases - their needs are very different from people who are only using Python for small scripts.
On the other hand async functions can be very cheap.
Again, which useful property does async protect?
Also, I've found that ChatGPT/Claude3.5 are much, much smarter and better at Python than they are at C++ or Rust. I can usually get code that works basically the first or second time with Python, but very rarely can do that using those more performant languages. That's increasingly a huge concern for me as I use these AI tools to speed up my own development efforts very dramatically. Computers are so fast already anyway that the ceiling for optimization of network oriented software that can be done in a mostly async way in Python is already pretty compelling, so then it just comes back again to developer productivity, at least for my purposes.
It will be interesting to see how this goes over the next few years. My guess is that a lot of lessons were learned from the python 2 to 3 move. This plan seems pretty solid.
And of course there's a relatively easy fix for code that can't work without a GIL: just do what people are doing today and just don't fork any threads in python. It's kind of pointless in any case with the GIL in place so not a lot of code actually depends on threads in python.
Preventing the forking of threads in the presence of things still requiring the GIL sounds like a good plan. This is a bit of meta data that you could build into packages. This plan is actually proposing keeping track of what packages work without a GIL. So, that should keep people safe enough if dependency tools are updated to make use of this meta data and actively stop people from adding thread unsafe packages when threading is used.
So, I have good hopes that this is going to be a much smoother transition than python 2 to 3. The initial phase is probably going to flush out a lot of packages that need fixing. But once those fixes start coming in, it's probably going to be straightforward to move forward.
AMD EPYC 9754 with 128-cores/256-threads, and EPYC 9734 with 112-cores/224-threads. TomsHardware says they "will compete with Intel's 144-core Sierra Forest chips, which mark the debut of Intel's Efficiency cores (E-cores) in its Xeon data center lineup, and Ampre's 192-core AmpereOne processors".
What in 5 years? 10? 20? How long will "1 core should be enough for anyone using Python" stand?
There is code that may benefit from the free threaded implementation but it is not as often as it might appear and it is not without its own downsides. In general, GIL simplifies multithreaded code.
There were no-GIL Python implementations such as Jython, IronPython. They hadn't replaced CPython, Pypy implementation which use GIL i.e., other concerns dominate.
If you're looking for a 32x or 128x performance improvement from python supporting multi-core you should probably rewrite in C, C++, Rust, or Fortran and get that 100x improvement today on a single core. If done properly you can then ALSO get the gain from multiple cores on top of that. Or to put it another way, if performance is critical python is a poor choice.
> "Or to put it another way, if performance is critical python is a poor choice."
To put it another way, just because performance isn't critical, doesn't mean that more performance for free is not desirable or beneficial, or that ignoring 127/128ths of available performance is fine.
A piece of code takes 6h to develop in C++, and 1h to run.
The same algorithm takes 3h to code in Python, but 6h to run.
If I could thread-spam that Python code on my 24 core machine, going Python would make sense. I've certainly been in such situations a few times.
Julia is one that is gaining a lot of use in academia, but any number of modern, garbage collected compiled high level languages could probably do.
(And as a side note: I have never, in around a decade of writing C++, heard std::thread described as "easy to debug.")
I expected that dropping down to C/C++ would be a large jump in difficulty and quantity of code, but I've found it isn't, and the dev experience isn't entirely worse, as, for example, in-editor code-intelligence is rock solid and very fast in every corner of my code and the libraries I'm using.
If anyone could benefit from speeding up some Python code, I'd highly recommend installing cppyy and giving it a try.
To be honest, I don't know much about the speed, as my use-case isn't speeding up slow code.
Every DL library comes with its own C++ backend that does this for now, but it's annoyingly inflexible. And dealing with GIL is a nightmare if you're dealing with mixed Python code.
in Python language specifically. Their library may have already done some form of parallelization under the hood
There are a lot of simple cases where multi-threading can easily triple or quadruple the performance.
I used to write a ton of MPI based parallel python. It's pretty straightforward. But one could easily imagine trying to improve the multiprocessing ergonomics rather than introducing threading. Obviously the people who made the choice to push forward with this are aware of these options, too. Still mildly puzzling to me why threads for Python are needed/reasonable.
But the mai point against multiprocessing seems to be that spawning new processes is slow ...
That single "alternatives" paragraph doesn't answer at all why mp isn't viable for python level parallelism.
I am no longer invested in python heavily, I am sure there are discussions or documents somewhere that go into this more. Might be that it's simply that everyone is used to threads so you should support it for sheer familiarity. All I am saying is it's not obvious to a casual observer.
Since Python leads over any other languages in the ML community, and ML is a hot topic right now, it makes sense for Python developers to secure the lead by making the life of ML developers easier, which is by introducing GIL-less multi-threading.
IDK what l should and shouldn't be written in, but there are a very large # of proud "pure Python" libraries on GitHub and HN.
The ecosystem seems to even prefer them.
Why shouldn't someone who prefers writing in python benefit from using multiple cores?
I did use the words "most things". I'm not saying this is a bad development for Python, or that nobody should use it. But if performance is a top priority, Python is the wrong language and always has been.
I use Python from time to time, it's fun and easy to put certain kinds of things together quickly. But each time I do a project with it, the first thing I ask myself is "is this going to be fast enough?" If not I'll use something else.