Hacker News Reader

The thing is, there's a lot more nuance to it than this. Async/await is part of the language syntax in python, but asyncio is only one particular implementation of an event loop framework to power it. But really what async/await provides is a general-purpose cooperative multitasking syntax. This allows other libraries to implement their own event loop frameworks, each with their own different semantics and considerations (the two best-known alternatives being Curio and Trio). At a language level, there's nothing even forcing you to use async/await for ascync IO -- you could, if you really wanted, probably write a library that used it to start threads and await their completion.

So you have, from highest-level to lowest-level: application code, async/await language syntax, the event loop framework, and then the implementation of the event loop itself. The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level.

But that means that even if you do "async all the things", you'll only encounter this situation if you write your application code in a particular way. It just so happens that "in a particular way" is, in this case, the overwhelming majority of how people write it, which is, of course, why the OP article is relevant.

> The OP article concerns a peculiar implementation detail in the lowest level that makes it very easy to write bugs at the highest level.

Are other async implementations using the asyncio.Task abstraction? I haven't looked into it, but I assumed that asyncio.Task was tied to the asyncio implementation and event loop.

asyncio.Task is part of the asyncio event loop framework. So any event loop implementation that conforms to that framework will have to have one (including the default event loop implementation that ships with asyncio). So for example, uvloop, which is an alternative event loop implementation that works with the asyncio event loop framework, also uses asyncio Tasks.

Other event loop frameworks can do whatever they want, and presumably, wouldn't be importing from asyncio. Whether they have a similar abstraction is completely up to the framework itself. Trio, for example, doesn't have a concept of a task object at all, because it enforces a strict tree structure for tasks.

> This allows other libraries to implement their own event loop frameworks

At work someone replaced the default library with another faster implementation.

Then the unix socket listener task was not working.

A few hours of git bisect later, I found out the offending commit was the 1 line switching the event loop. Seems the fast implementation didn't implement unix sockets and just had "pass" in the function.

I am a huge fan of parallel and async code. I spend a lot of time researching it and trying to design software that is easily parallelisable.

Many GUIs use the event/message pump pattern, such as Windows 32 API. Qt does something with its event loop (QEventLoop)

Threads are a rather low level instrument to get background tasks going because the interface between the main thread and the threads is rather omitted.

In Java you could use a ConcurrentLinkedQueue. And in Python you can use JoinableQueue.

I am heavily interested in this space because I want to write understandable software that anybody can pick up and work with. I worked on a JMS log viewer that used threads but would crash with ConcurrentModificationException due to not being thread safe. I changed it to be thread safe but its performance dropped through the floor. In my learnings since then I should hast sharded each JMS connection topic to its own thread or multiplexed multiple JMS topics per thread and loop over them. The main thread can interrogate the thread with a lock, that should be faster than every thread trying to acquire the lock. It would be driven by the main thread but the work is done in the background. The threads can keep the fetched messages in memory until the main thread is ready for them.

I think with the right abstraction, thread safety can be achieved and concurrency shouldn't be something to be afraid of. It is very difficult and challenging working at the low levels of concurrency such as a concurrent browser engine. (I've not done that though.)

This is why languages such as Pony lang, Inko, Cyber and Erlang, Elixir are so promising. We can build high performance systems that parallelise.

Writing an async/await pipeline that looks synchronous is far easier to understand and maintain than nested callbacks. So I can see where async is useful. I just hope we can design async software to be simpler to maintain and extend.

> Where I'm particularly unconvinced of their use is in server side view and api end point processing. The majority of the time you have maybe a couple of IO opps that depend on each other. There is often little than can be parallelised (within a request) and so there are few performance gains to be a made. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug. Traditional synchronous imperative code run with a multithreaded server is proven, scalable and much easier to debug.

Python doesn't have multithreading that scales or supports real parallelism. asyncio has very measurable performance benefits for exactly that use case you've mentioned versus threaded servers.

Sorry that's not accurate. Asyncio and threading offer the same variety of "parallelism" , which is that both can wait on multiple io streams at once (the gil is released waiting on io). Neither offer CPU parallelism, unless lots of your CPU work is in native extensions that release the gil. In that unusual case, threading would offer parallelism where asyncio wouldn't.

Asyncio's single advantage is you can wait on lots of io streams, like many thousands, very cheaply without having to roll non blocking IO queueing code directly.

I didn't say that asyncio offered parallelism, I'm pointing out that normal assumptions about multithreading you'd make with other languages don't always apply to Python. You'd typically assume that threads offer parallelism, a property you might choose to use them for over something like single-threaded asyncio.

I've found that for even IO bound workloads, the amount of throughput plateaus when using a relatively small amount of threads despite the GIL being released on IO.

sorry for misreading that from your post! my own benching with threads vs. asyncio has never found any performance difference between the two approaches (asyncio slightly slower). if you need very wide throughput, then yes asyncio is better. otherwise, it's very difficult to create equitable comparisons between threaded and asyncio code.

Haha, it's cool. I once had a problem where I needed to scrape a bunch of sites at once, and ended up using it as a benchmark for threads vs asyncio. The overhead with threads was high even when using a handful of threads, and quickly hit a plateau after scaling to 4-8 threads on a 16 core machine. I was a big fan of the concurrent.futures executor model and generally thought it was good performance anyway, but figured I'd try to implement it in asyncio, and rate limited both for a fairer comparison. It was so shockingly fast that it was the last project I reached for threads again with on Python. I can't say that it was an equitable comparison, but it certainly left an impression on me.

it was probably whatever concurrent.futures does. I use plain threads and get great results for anything under 25 threads on a small laptop. you always keep the same set of threads running in a pool and use a decent queueing model.

certainly, doing that kind of thing with asyncio likely has a lower learning curve because you don't need to worry about "pools of workers" so much. concurrent.futures should do that for you also but that would be the first place I don't trust

> GUI toolkits (like Textual) however are a really good use case for Asyncio.

Only if the GUI toolkit is explicitly written to be asyncio-aware and use asyncio's event loop. Textual appears to be written specifically to do that.

However, other GUI toolkits that I'm aware of that have Python bindings aren't written that way. Qt, for example, uses its own event loop, and if you want anything other than a GUI event to be fed into Qt's event loop so your event-driven code can process it, you have to do that by hand and make sure it works. There is no point in even trying to use another event loop, such as Python's asyncio event loop, since that loop will never run while Qt's event loop is running.

There's a qasync library that I've used to bridge asyncio and Qt, it works pretty well. But yes, you need some extra support.

>Where I'm particularly unconvinced of their use is in server side view and api end point processing.

Sure, performance isn't going to get better, but for websockets and server sent events the occasional long-lived async task can be great. Especially when you need to poll something, or check in on a subprocess.

These days with graphql, or complex microservices architectures, you could have multiple hops to fulfil l the original request.

Flask sync will hold that thread hostage until the request is done. Where async with properly used async libs will allow other requests to process.

We often have medium sized reports take seconds. That is a lot of time to wait. And would just end up bloating your service scaling to handle more connections.

Any service with decently long lived network requests will benefit from event loop handled scheduling.

I don't think he was referring to python when he mentioned server-side, but to real multi-threaded runtimes.

I don't write any colored function code in python, I'd much rather work with process/thread pools

Me too, but threading is botched in Python. Not just the Global Interpreter Lock. Some Python packages are not thread-safe, and it's not documented which ones are not. Years ago I discovered that CPickle was not thread safe, and that wasn't considered a problem.

Since the GIL is there and it restricts multithreaded issues to happen, why would python packages be thread-safe?

You can still have thread safety issues with the GIL in place, because globals and other data is shared between threads.

For example, you can put a dictionary at the module level, thread A can set a key in that dictionary like “name”, thread B can overwrite it, and then thread A comes back, does dct[“name”] and gets an unexpected answer.

This is a relatively easy mistake to make, a lot of python code has module level variables.

I really don't get the whole coloured function thing. How's it not just the function signature? You might as well claim a new function argument makes a new colour. Granted, all of my use of async is in Rust in which the compiler picks this stuff up, so maybe in python there are other concerns I'm missing.

Honestly, I think the whole colored function thing is BS. I've never run into an issue with it.

I think the "new function argument makes a new colour" is accurate on several levels. async/await is a dual for streamlining working with the Promise or Future or Task Monad (however you want to call it). (And async/await syntax in most of the languages that have it can actually be [ab]used for a substandard "do-notation" for nearly any Monad you want to use.)

At face value, yeah, every function in Haskell that accepts an IO monad is now "IO colored", but at the same time, that's a silly way to look at it. It's just extra type information and type bindings flowing as types flow through functions. It's just a bit of a tautology that functions that deal with other functions that need that type need to deal with that type themselves.

Functions that use Maybe/Option/nulls are all "nullable colored". Functions that use or return integers are clearly "integer colored". That's what programming languages do: they try to track how your types flow through functions. "Coloring" is a bad metaphor or at least a useless one, we just call that "types". Admittedly, I think that's why Python and JS users predominantly use the "what color is your function" complaints the most because all of the rest of typing information for them is generally opt-in and easily ignorable/forgotten.

I am not sure I agree that the GUI is a good use case for async. A human interaction with the program must almost always pre-empt whatever the program was running, so I can not see how a cooperative multi-threading runtime like async Python can work in such a scenario.

Well it does work so "it can't work" isn't a very substantial criticism.

Is there some nuance or detail that you meant to include?