Hacker News Reader

Comment by kissgyorgy | Hacker News Reader

After 2 years of using asyncio in production, I recommend to avoid it if you can. With async programming, you take the complexity of concurrent programming, which is way harder than you can imagine.

Also nobody mentions this for some reason, but asyncio doesn't make your programs faster, in fact it makes everything 100x SLOWER (we measured it multiple times, compared the same thing to the sync version), but makes your program concurrent (NOT parallel), where the tradeoff is you use more CPU, maybe less memory, and can saturate your bandwidth better. Never do anything CPU-bound in an async loop!

View on HN

Asyncio is not ment for CPU Bound Task but for IO Bound Tasks. You should have used multiprocessing. The problem you describe is exactly what asyncio is used for: saturate your bandwidth better.

maybe the right aproach would have been a Threadpool? Plus you don't have to refractor the task. Just let it run sync, but you can make it also async

"Asyncio is not ment for CPU Bound Task but for IO Bound Tasks."

You can amplify this; pure Python is not meant for CPU-bound tasks. If you have a pure Python task, you've CPU optimized it somewhat, and it's still too slow, the answer is, get out of pure Python.

There's half-a-dozen options that still leave you essentially in Python land (just not pure Python anymore) like cython or NumPy, and of course dozens of other languages that leave Python entirely.

To accelerate a pure Python task to the speed that you can get out of a single thread of a more efficient language, you have to perfectly parallelize your task across upwards of 40 CPUs (conservatively!), because that's how slow pure Python is. asyncio isn't a solution to this, but neither is any sort of multiprocessing of the pure Python either.

This is not criticism. This is just engineering reality. Pure Python is a fun language, but a very slow one.

In other languages the async paradigm work for multiple kind of workflows, not just heavily IO bound ones.

But it only does so because you can move out that extra work from the async thread into a thread pool.

In Python there's no benefit in doing this due to the GIL, unless you're using a module which implements its own multithreading (for example in C). Python is not alone with this issue.

In other languages you're basically stepping out of the async paradigm in order to use threading in parallel. You can wait for result of the thread in the async loop without blocking.

I really enjoy using asyncio in Python for things where I have to do a lot of stuff in parallel, like executing remote scripts in a dozen of servers in parallel via AsyncSSH or for low workload servers which query databases.

In any case, what keeps me hooked on Python like a junkie is the `reload(module)` which I invoke via an inotify hook every time a file/module changes:

server.py

server_handler.py

reloader.py

server.py loads reloader.py (which sets up inotify) and that one takes care of reloading server_handler.py whenever it changes. server.py defers all the request/response handling to server_handler.py which can be edited on the fly so that the next request executes the new code. Instant hot reloading.

This also works very nice with asyncio (like aiohttp) and one ends up with very readable code.

If you need performance, then use Java, Rust, Go or C/C++, but for prototyping and tooling I absolutely love this approach with Python.

Can you give me an example which languages do this?

In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside.

So I've also used coroutines in Kotlin for CPU bound tasks with a nice speedup, where multiple sub-tasks get executed in parallel and then gathered when needed to produce new tasks etc., in a granular way that would be very intrusive doing with threadpools. Here I basically took the existing coroutine code and slapped some more threads on it.

With that said, I'm not entirely sold on Kotlin's way either. Both Kotlin and Python have the "what color is your function" problem.

> In Kotlin for instance you can execute your coroutines with different "dispatchers", which have different behaviors. You can for instance use a dispatcher with multiple threads in a threadpool, run it in the main thread, a specific thread, in threads with low/high priority etc. Basically allowing you to write code using the async/coroutine paradigm, but then control its execution from the outside.

I'm not very experienced with Kotlin, but that sounds similar to python's run_in_executor [1]

[1] https://docs.python.org/3/library/asyncio-eventloop.html#asy...

Look at seastar.io which is an async concurrency library for C++. It is painful in that it resembles node.js but it is very performant. I'd be interested in seeing a version adapted to use C++20 coroutines.

Depending on the specifics, a threadpool proabably wouldn't help with a CPU bound task due to GIL limitations. `multiprocess.Pool` might (though process pools aren' without their overheads)

any concurrency system that is properly designed supports both types of tasks.

I only know C#: https://learn.microsoft.com/en-us/dotnet/csharp/async#recogn...

There you also have to act differently based the task

The docs on that page explicitly say you just have to provide a different parameter (basically, to force it to use a thread-like instead of select-like approach).

The docs on that page also point you to a different task library which does what I'd expect:

"If the work is appropriate for concurrency and parallelism, also consider using the Task Parallel Library."

I checked those docs and that is a framework that makes sense.

Not sure how that's different than Python asyncio's "run blocking stuff with `run_in_executor`, the result (future) of which integrates into the async event loop?

it's threaded with the option to use 1:M, N:N or N:M. It's a mental model that includes both asyncio and threads, while async will always only be for IO-blocking calls that can be multiplexed.

This just sounds like you fundamentally misunderstood how asyncio works and applied it to a use-case which it wasn't suited to. Why would you use async if you're doing a bunch of CPU-bound work?

This is an insane take, if you are doing async you already HAVE a need for concurrent programming. Async just makes that simpler to read and write.

Unfortunately the majority of people who use asyncio in the real world do not have a need for concurrenct programming. They are drawn to it because they believe it is faster, which, generally, it is not. That isn't to say there aren't reasonable usecases - but most people using asyncio are writing webapps which spend a lot of time in the CPU generating or outputting JSON/GraphQL/HTML.

I think this may be influenced by Node. With JS and Node, async is cheap and ergonomic, so it's very widely used, even if the latency gains are marginal.

With Python and asyncio, neither assumption holds, but the idea of shaving off some latency persists: if we rewrite everything using asyncio, our DB accesses can be parallelized, yay! This may or may not be a net win in latency; reworking your DB access patterns may gain you more.

> reworking your DB access patterns may gain you more.

Nope: https://techspot.zzzeek.org/2015/02/15/asynchronous-python-a...

I don't say that asyncio gives you nothing! Go for it with your DB access if you have already gone async in your design.

But often the lower-hanging fruit is doing more joins in the DB, fetching fewer columns, and writing your query more thoughtfully. Async only helps if you can do something while waiting for the DB to complete your query. A common anti-pattern, exacerbated by ORMs, is doing a bunch of small queries while passing bits of data between them in Python; moving that to asyncio won't help much, if any.

> But often the lower-hanging fruit is doing more joins in the DB, fetching fewer columns, and writing your query more thoughtfully.

Reluctant to add a "me-too" comment but I think you have it there. For whatever reason people are much keener to investigate different ways of scheduling IO than they are to investigate what the query plan looks like and why.

It may not be the norm, but there are definitely apps/services out there that require heavy-duty database queries and can have their throughput bottlenecked by synchronous DB IO latency, and would benefit from asyncio on the database side. Also, SQLAlchemy supports asyncio now.

> Async just makes that simpler to read and write.

Really? I definitely had a huge struggle reading async python, because generators and yields break the concept of "what is a function" and struggled writing async python because of function coloring.

I am really not a fan of generators and yields for the same reason. But over time I've come to see it as ever-so-slightly more elegant syntax sugar for a class that maintains internal state to sequentially return specific values in response to a series of calls to a function.

The least they could have done is sugared it to something that isn't "def", like "gen" or "defgen"

Unless I'm misunderstanding your comment they do, it's `async def` to denote a coroutine.

They're referring to generator functions. A generator function typically open a resource and then use a for loop to read (for example) a line at a time and yield it. once you yield, control returns to the caller (like a return) but subsequent calls to next on the returned expression pick back up at the yield point. So function calls maintain state-behind-the-scenes. It's not just a simple call-and-return model anymore.

To me this violates a core principle: the principle of least surprise. It completely changes how I have to reason about function execution, especially if there are conditionals and different yields.

It's the buried `yield` in a loop somewhere, magically changing your function into a generator, that can be confusing – it's on the original coder to have the docstring say `"""Generator yielding records from DB."""` or something.

Day to day Python async should be using generators and yields.

Generators are great for predictable concurrency. Things like asynchronous I/O, with timeouts and stuff, require more support on the language and stdlib level.

Async python by design is not concurrent, nothing is executing at the same time on different cores/threads. There's one worker loop and one task running on it at a time.

Concurrency is not Parallelism. You should watch this quintessential talk about the difference - https://www.youtube.com/watch?v=oV9rvDllKEg

This is Rob's perspective, but it's not universally shared. In my mind, all concurrency is a form of parallelism (parallel tasks, but not parallel threads) while not all parallelism is concurrency. I have frequently solved concurrency problems with thread pools, rather than using non-blocking IO, because the programming paradigm is a lot simpler.

You got the the wrong way round: all parallelism is a form of concurrency.

In Python, thread pools and async work mostly the same in practice, except that threads have a bigger overhead. So they can be used to solve the same problems.

I think you mean it is not parallel. Cooperative multitasking is absolutely a type of concurrency

I'm pretty sure a lot of people don't understand the tradeoffs and don't really need concurrent code. There are really only a couple of use-cases where it's really handy: for example an API gateway, where you only get and send HTTP requests. Other than that, I don't see how it worth the insane complexity (compared to sync code).

You can write web servers in a sync manner; Flask, Django is way better if you only need an API, choosing FastAPI for a simple REST API is a huge mistake.

> FastAPI for a simple REST API is a huge mistake

FastAPI can be sync or async, although sync will have "a small performance penalty"

https://github.com/tiangolo/fastapi/issues/260#issuecomment-...

Choosing fastapi for a fast api is a mistake?

Choosing unnecessary complexity for a simple task is a mistake. FastAPI is not faster than Django or Flask for a single request, quite the opposite.

There are other valid reasons than speed:

https://christophergs.com/python/2021/06/16/python-flask-fas...

How is Django less complex than FastAPI? Django was built for monolithic web applications - it does fine with REST, but your position is backward. FastAPI is purpose built for smaller APIs

What kind of APIs are you writing that don’t involve db lookups or calls to other services?

Asyncio does not make your db calls or io faster, it does make your code more complex.

The only thing async io gives you is scalability of many MANY concurrent io operations. You have to either be pushing seriously large traffic or doing a lot of long running websocket/sse/long polling type requests.

The only other use case is lots of truly concurrent io within one request/response cycle. But again that is unusual, most apis have low single digit db queries that are usually dependent on one another removing any advantage of async.

> The only other use case is lots of truly concurrent io within one request/response cycle. But again that is unusual, most apis have low single digit db queries that are usually dependent on one another removing any advantage of async.

Why just "within one cycle"? What about situation where you spend 1s in single db query and are 100ms cpu bound during request handling?

Transitional multithreaded Python web servers handle that very well and have for decades, you shouldn't need to manually handle yealding in simple cases in your application code.

The point is asyncio is to have find grade control of yealding.

This isn't a problem specific to python though. You hint at it, async programming (single thread worker loop) is not the right way to deal with CPU-bound tasks. NodeJS or any other async language would have the same problems. You want multiprocessing or multithreading, where work is distributed to different CPU cores at the same time. Python gives you the ability to use any of those three paradigms. Choose wisely.

After using gevent for about a decade I have started using asyncio for new projects, just because it's in the standard library and has the official async blessing of the Python gods. Indeed it is way harder. I'm always coming up against little gotchas that take time to debug and fix. Part of me enjoys the challenge of learning something new and solving little puzzles. It's getting easier, especially as I build up a collection of in-house async libraries for various things. As for performance, it's not too bad for mostly io bound tasks, which is why one uses async in the first place. Some tight loop benchmarks for message passing with other processes show it to be about half the speed of gevent in my case, which is fine. It's nice to be able to deploy async microservices without installing gevent, and there's a certain value to the discipline that it imposes. I like how I am able to bring non-async code into the async world using threading. I imagine the performance would improve quite a bit with pypy, perhaps exceeding that of gevent. Gevent makes it so damn easy, I've been spoiled. I was disappointed when asyncio came out, as I would have preferred the ecosystem moved in the gevent direction instead; but I'm coming around. It's super annoying how the python ecosystem has been bifurcated with asyncio. You really have to choose one way or another at the beginning of a project and stick with it.

And yeah, async programming (in Python) isn't really for CPU bound stuff. You might benefit from multiprocessing and just use asyncio to coordinate, which is what it excels at. PyPy can really help with CPU bound stuff too, if the code is mostly pure Python.

Java with Project Loom will be like gevent.

My opinion is that async/evented is a (useful) performance hack.

You wouldn't do it that way for any reason other than you can get better performance than way than by using threading or other pre-emptive multitasking.

It's not a better developer experience than writing straight forward code where threads are linear and interruptions happen transparently to the flow of the code. There are foot guns everywhere, with long running tasks that don't yield or places where there are hidden blocking actions.

It reminds me a bit of the old system 6 mac cooperative multitasking. It was fine, and significantly faster because your program would only yield when you let it do it, so critical sections coule be guaranteed to not context shift. However, you could bring the entire machine to a halt by holding down the mouse button, as eventually an event handler would get stuck waiting for mouse up.

Pre-emptive multitasking was a huge step forward -- it made things a bit slower on average, but the tail latency was greatly improved, because all the processes were guaranteed at least some slice of the machine.

"Performance" is too wide a term, asyncio does not improve performance in the most widely understood interoperation, "speed". Talking about performance benefits of asyncio causes people to misunderstand where it is best used.

"Scalability" is a better word to use when talking about asyncio. Along with describing complex concurrent programming such as for a GUI, where the added syntactical complexity if outweighed by the reduced boilerplate of traditional GUI programming.

Yeah, talking speed in the manner of C1M sorts of things. Server stuff - where python tends to be rather than GUI side. It's demonstrably faster for IO to not be context switching with threads -- but if threads were equal weight to events, I don't the event programming style being more programmer productive than the threaded style. I can see where events are useful in a GUI context, but UI Thread + thread pool dispatch still works well, if your framework supports it.

Just this week, I got to the bottom of a performance issue in django because the developers were using async, and then doing eleventy billion db queries to import a big csv, thereby blocking all other requests.

One of the questions I ask on our programming interviews is "how would you make this (cpu bound) thing go faster" -- Async is definitely a low quality answer to that, when things like indexing or hash lookup vs array scan are still on the table.

> 100x SLOWER (we measured it multiple times, compared the same thing to the sync version)

I'd need to know the methodology. Asyncio is "gated", meaning that coroutines execute in tranches. When you queue up a bunch of coroutines from another coroutine, they don't execute right away instead they go in a list. Then when the current tranche completes the next one starts. There's some tidying up which occurs with every tranche.

As an (only one) example of "measuring the wrong thing", if you queue up a single task 1000 times which takes as its argument a unique timer instance it matters whether you queue them up one at a time as they finish running or queue them all up at once.

    #!/usr/bin/python3
    # (c) 2022 Fred Morris, Tacoma WA USA. Apache 2.0 license.
    """Illustrating the tranche effect in asyncio."""

    from time import time
    import asyncio

    N = 1000

    class TimerInstance(object):
        def __init__(self, accumulator):
            self.accumulator = accumulator
            self.start = time()
        def stop(self):
            self.accumulator.cumulative += time() - self.start
            return
        
    class Timer(object):
        def __init__(self):
            self.cumulative = 0.0
            return
        def timer(self):
            return TimerInstance(self)

    async def a_task(timer):
        timer.stop()
        return

    def main():
        loop = asyncio.get_event_loop()
        timing = Timer()
        overall = time()
        for i in range(N):
            loop.run_until_complete( loop.create_task( a_task(timing.timer()) ) )
        print('Sequential: {}   Overall: {}'.format(timing.cumulative, time() - overall))
        timing = Timer()
        overall = time()
        for i in range(N):
            loop.create_task( a_task(timing.timer()) )
        loop.stop()
        loop.run_forever()
        print('Tranche: {}   Overall: {}'.format(timing.cumulative, time() - overall))

    if __name__ == '__main__':
        main()


    # ./tranche-demo.py 
    Sequential: 0.02229022979736328   Overall: 0.04146838188171387
    Tranche: 6.084041595458984   Overall: 0.012317180633544922

Async in general (not just python) doesn’t speed up any computation.

The value is allowing you to do something else while waiting for IO to finish. If you are constantly doing IO, async is a good tool. Otherwise, it won’t help.

You are doing it wrong then, asyncio is never designed for and not useful for CPU bound tasks. Use multiprocessing for CPU bound tasks. In our system before asyncio, sqlalchemy commits and httpx requests are blocking The API calls and it takes 21 seconds every time, now only 1.2 seconds. We had optimized by making every IO (db.commit(),httpx.post) to run inside asyncio.create_task.

So they are operating outside of the request without slowing it down.

We had used aync since 2014 ( twisted, tornado), and officially having async await in python starting 3.7 was the best thing happened to python.

The clue is in the name: asyncIO is for, well, I/O bound stuff...

asyncio gets a lot of hype, but I prefer concurrent.futures. It feels more simple and general since I don't have to make everything awaitable.

It used to get brought up every time asyncio was mentioned on here.

I either use multiprocessing, or celery. using async/await looks ugly, doesn't solve most problems I would want it to, and feels unstable.

Maybe once development around it settles down I'll revisit, but I can't imagine wanting to use it in it's current form.

I've experienced something similar using asyncio as well, but I think you're wrong to blame the inherent complexity of concurrent programming.

I've used curio as an alternative concurrent programming engine for a med-large project (and many small ones). In comparison to asyncio, it's a joy to use.

What I'm saying if you don't need concurrency, don't trade off simplicity. For REST APIs and websites, probably not needed.

Rest APIs and websites are IO bound. Your python program is there purely coordinating between disk io, database (network) io, calls to other services (io). You can be choreographing thousands of requests concurrently and still barely doing any cpu.

The overhead to do a similar service using threadpools is much higher and the reason asyncio exists.

Reply on news.ycombinator.com