Guide to Concurrency in Python with Asyncio
integralist.co.uk
integralist.co.uk
Edit: elsewhere on HN right now:
It's a much simpler model that's just as powerful, and makes it easy to get things right. It eliminates the concepts of futures, promises, and awaitables. It has only one way to wait for a task: await it.
For a theoretical explanation of it's "structured concurrency" model see the now-famous essay https://vorpus.org/blog/notes-on-structured-concurrency-or-g...
I've been stung, time and again by asyncio.create_task() swallowing exceptions[1], and functions not telling callers about background tasks[2].
For others looking to NOT rewrite all their code using a 3rd party event loop, here's a simple reproduction of nursery's behaviour in raw asyncio -
Hopefully, this will help lift the curse of asyncio :)
[EDIT] A library is also available - https://github.com/Tygs/ayo (with amazing operator overload)
async def main():
async with Nursery() as ny:
ny.start_soon(a_1(), a_2(), ..., a_n())
ny.start_soon(b_1(), b_2(), ..., b_n())
.
.
.
ny.start_soon(x_1(), x_2(), ..., x_n())
class Nursery:
def __init__(self):
self.tasks = set()
def start_soon(self, *coros: typing.Coroutine):
for coro in coros:
self.tasks.add(asyncio.create_task(coro))
async def __aenter__(self):
return self
async def __aexit__(self, *args):
try:
while self.tasks:
tasks = self.tasks
self.tasks = set()
done, pending = await asyncio.wait(
tasks, return_when=asyncio.FIRST_COMPLETED
)
try:
for task in done:
await task
finally:
self.tasks |= pending
finally:
for task in self.tasks:
task.cancel()
Luckily, Trio now exists. Trio has a consistent and simple story for cleanly handling errors and cancellations. Debugging is still a little difficult, but at least code written with Trio has radically fewer problems to debug.
Here's a link on the Trio design: https://trio.readthedocs.io/en/stable/design.html
There are some more great articles and talks, but I don't have any links at the ready, sorry.
It's also exciting to see that Java has taken the same view, and is well on its way towards delivering it (https://wiki.openjdk.java.net/display/loom), or even improving on it (https://wiki.openjdk.java.net/display/loom/Structured+Concur..., see also: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...)
[1] https://docs.python.org/3.8/library/concurrent.futures.html#...
The article didn't limit its focus to I/O, though. It felt more like an explanation of modern concurrency and asynchronous options in Python (besides just threading and and multiprocessing).
The only practical issue with the GIL in Python is that it forces you to use new (heavy) Python processes to parallelize computationally heavy tasks. Not so much of a concern depending on your application, but it is a gotcha for people new to Python.
The real issue, though, as your parent poster mentioned, is the number of different ways to access asynchronicity and concurrency in modern Pythons. It really does run counter to Python PEP20 [0] and can lead to some communication difficulties among developers.
I believe concurrent.futures is meant as a way for asyncio to spawn threads or processes -- which lets you essentially call sync code from async and not have it block the event loop.
From the docs[1] -
The concurrent.futures module provides a high-level interface for asynchronously executing callables.
The asynchronous execution can be performed with threads, using ThreadPoolExecutor, or separate processes, using ProcessPoolExecutor. Both implement the same interface, which is defined by the abstract Executor class.
[1] https://docs.python.org/3.8/library/concurrent.futures.html
Multi-process code can, of course, address some of these issues, but it comes at the cost of spinning up multiple OS processes and costly inter-process communication etc.
For one, regardless of the GIL it is still possible to have races in multithreaded python code. You have to worry about RMW because the threads are arbitrarily preemptible at a bytecode boundary. This is not the case with asyncio.
Additionally you can release the GIL when making calls to C code. Asyncio is giving you concurrency by using nonblocking calls, you can’t run two CPU bound threads in parallel with it. But you can do this with python threads. So it’s not true that neither can accomplish more than the other.
It's a confusion that has been quickly resolved every time a team mate of mine has run into the issue, but I have seen it happen multiple times (at different places).
There's very little glue between them. And none of it is duck typed, like the rest of Python. If I switch between them, I need to rethread the entire system.
Imagine a naive young scientist who has been "trained" to code in Matlab. He is not a computer scientist, of course, but he profits if his program goes fast, even before actual computer scientists get involved to make the thing proper. Indeed that may make the difference between the project getting off the ground or not going forward. Such cases have been known to exist.
Now, he comes across some code that can not be written in pre-optimized matrix routines, but nevertheless populates deterministic addresses in a container without any further side effects. It could wait for some IO, or perhaps it just does a thing that takes a while on the CPU core (like iterations of something with results getting saved somehwere). In any case, it's gotta go into a for loop and that's slow for most languages of that sort.
Given that he has a good number of cores, threads etc. available, he figures it'd be pretty cool if the thing run in parallel.
So he changes the "for" statement into "parfor". Matlab then starts a CPU pool and runs the loop to completion in parallel on both threads and cores without further issues. The whole thing just runs x times faster.
My point here is this: The whole thing in Python is complicated. It is so, because other languages, like Matlab, make it laughably easy. The Pythonic way would have been to offer such an easy option. However, if you are not well-versed in serious coding, then concurrent anything in Python is hella complicated. If you disagree, you likely know more about multi-x than the average person typing things into a computer.
I do not doubt that such a simple approach has its limitations. But it seems to me that in practice - and in particular when I look at most of the examples presented - such a simple way would have been completely sufficient. I do readily accept that I may be completely wrong.
edit: And now I realize that your verb was "debug" and not "write". Sorry.
In my experience, 90% of these novices haven't the faintest clue of what's going on.
"It worked before, but now I'm running out of memory."
"Why are you telling me that I'm crushing disk I/O?"
"I went from 1 core to 40, and now it runs slower!"
"I requested 32 nodes--why doesn't it run 32x faster?"
"I requested one core, but MATLAB saw the awesome stuff in /proc/cpuinfo and ran 40 cores. Why is it even slower than ever?"
Bless the clueless, for they shall inherit the earth. (But please, don't let them write coronavirus models in public...)
It took me 6 weeks to debug and fix. I did nothing but debugging at the most primitive level. Probably could have done it more efficiently, but I was young and inexperienced.
Worst and most boring 6 weeks of my life caused by concurrent debugging.
Making the event loop self-managed added a ton of clunkiness in aio apis (use-your-own-loop, loop lifetime management, etc) and becomes mentally complex for newcomers. Theres also the issue that these huge aio frameworks rewriting the same TCP clients have emerged only differentiated by their loop management patterns.
However, it's still my main language. Everything I end up making just ends up being written in Python because that's the language I know the best.
I could learn another language, and I have learned other languages, but Python seems to be the language that mentally clicks with me the most. I think it's because it's the first language I learned.
Suffice to say, there are so many things wrong with Python and they're all compromises that were made to appeal to wider audience. Unfortunately, I don't think that the "wider audience" is those who are looking for concurrency.
> Unfortunately, I don't think that the "wider audience" is those who are looking for concurrency.
I don't buy this. Many network heavy shops use Python so concurrency is relevant. The language is used in web, finance, cloud, data pipelining/etl, and so many more. Dropbox, Robinhood, Spotify, Instagram, Uber, Google, etc, are the "wider audience" and they certainly care about concurrency.
I imagine the problem is, that neither JS or Golang have to deal with, is Python supports plain old threads. If you have a single global event loop, and then you try execute a future what should happen?
1. Should every every thread have its own loop? Then how would you send futures to another thread.
2. Should the global loop only exist on a single thread? Then you would have to pay a synchronization tax every time you pushed a task to the loop.
Not saying the Python solution is 100% right (I've only thought about it for 5 minutes), but I can see how being compatible with threads has caused this situation. I believe the default now is you don't have to think about loops, and Python does the right thing for 99% of use cases.
You need to use a process pool in Python, thread pools are for I/O bound tasks and cannot use more than 1 core.
I find 2 of those to be easier to reason about and all to be superior.
The OP was commenting on what in his opinion was the easiest. That explicitly represents a judgement of Python's features compared with those offered by other languages.
Are we supposed to only praise Python in threads about Python?
Python seems to have lost its way after the 2 to 3 shift and is no longer what I'd consider "pythonic".
Guido was competent, but not nearly as brilliant as, for example, Larry, but Python 2 was dramatically better than Perl. I think this is mostly by chance. Hundreds of new languages come out every year, and by whatever fluke, a few of them really work. Python 2 was that.
In an ecosystem like that, past performance is no indication of future performance. There are very few language designers who did well more than once (Guy Steele is the only one I can think of who wasn't a one-hit-wonder).
So I too felt like Python 3 was more of a step backwards than forwards linguistically. But at this point, it has features I need which Python 2 lacks, so I'm mostly working in Python 3. But it definitely feels (unnecessarily) less clean.
EDIT: coming from someone who has had great experiences with FastAPI and Uvicorn combined with Dask and Redis.
import multiprocessing
some_data = {1: "one", 2: "two"}
func = lambda: some_data.get(2)
process = multiprocessing.Process(target=func)
process.start()
Why?
I mean, come on. Who in their right mind would ever claim that Go is not a good choice to develop web services?
And are we supposed to turn a blind eye to Python's problems such as sub-par performance and the GIL?
I understand how Python fanboys will pick Python over alternatives whether that makes technical sense or not, but asserting that any compiled language cannot be used to deliver software that has been implemented with Python, and that explicit support for OO is a relevant requirement, is something that flies in the face of reason.
Asyncio on the other hand, feels like being beaten with a stick. It's so complicated that normal humans are never really going to understand it all.
On top of that, it's been morphing all through the 3.0 series, so there isn't even one spec to learn--it's more like a set.
Overall I understood little to nothing from your comment.
The async module has been changing, which like you, I dislike. The "right" way to do asyncio would have been to properly mull the design into its final form and then introduce it into Python3. Instead, it appears to have been dribbled in piecemeal over several releases.
Beyond that, it's just plain complex and confusing. I haven't bothered to use it for anything so far. I'm hoping someone will show up with a machete and cut it down into something usable. Right now, it resembles C++.
I'm not saying software can't be written with asyncio, I'm just saying curio and trio are better.
Besides the standard way, you can also register routes similarly to DJango and Flask: https://docs.aiohttp.org/en/stable/web_quickstart.html#alter...
[1] once memorably dubbed "the people's champion" in the introduction to one of his PyCon speeches.
[1] https://www.youtube.com/watch?v=Y4Gt3Xjd7G8&list=PLKLmIMXskJ...
Can anyone recommend a guide which explains how to actually do I/O using asyncio: how to accept connections, how to write to and read from a socket?
Obviously it is, or this wouldn't be post #800 that promises to finally make asyncio clear to newcomers.
Twisted sucks. It was a joke. Now it's been tacitly blessed to the point you can sprinkle the `async` keyword all over the place. Good luck. I hope you don't forget about any spots.
I truly think part of the reason Guido left is because he couldn't defend this shit.
No one in these comments are offering any palatable alternatives. A compiled language with no OO is not a replacement for Python's Asyncio.
Everything else about it is solid as hell.
Pathlib comes from Twisted IIRC.