Hacker News Reader

Comment by xwowsersx | Hacker News Reader

I'm not in a much of a position to evaluate Python's asyncio since I really have not used it very much. However, over the last few days, I started to dig into it and tried to get some (what I think are) very basic examples working and really struggled. That alone is not fully dispositive because I've used many async implementations in various languages and each of them have a bit of a learning curve and their own wrinkles you have to ramp up on. That said, my limited experience at least tells me that Python's async has a FUX that leaves a lot to be desired. I've found other languages make it a lot easier to do concurrency, parallelism, async vs sync, blocking vs non-blocking and all that. I don't really know if the issue is poor documentation, the semantics of the APIs themselves...really not sure. I do know that I'm left with the feeling that "this seems like it's going to be a pretty big PITA".

View on HN

Asyncio is annoying, and often unexpectedly slow. You think things are parallel, but then one misbehaving coroutine can hog your cpu bringing everything to a halt. GIL makes it useless for anything other than _heavily_ IO bound tasks.

And yeah, Python's documentation is useless. Never how to use stuff, only listing of everything that's possible to do / the API. Unfortunately that style is being mimicked by most other Python projects as well.

I think the big point of interest will come with Python 3.12 where it's proposed that as part of the faster CPython project, the GIL-per-process will be removed.

It IS easy.

The Python project just don’t make it obvious how to do it easy.

Right. So you think it's mainly an education/documentation issue? I'm totally open to that in fact being the case. One really simple use case I had was just annotating a method to run async and then calling that in a loop elsewhere which did speed up a bunch of work I was doing pretty dramatically.

  import asyncio
  def background(f):
      def wrapped(*args, **kwargs):
          return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs)

      return wrapped

  @background
  def my_io_bound_function():...

You need to run the executor somewhere ( loop.run_forever() or loop.run_until_complete() ), that can be in the current thread or a separate thread, keep in mind Python is still conceptually a single core.

Things I've found particularly useful:

* Optional / debug logging of coroutine start/exit.

* Stats logging, including count, runtime, request queue (multiple instances of the same function) depth.

* Printing tracebacks within the coroutine context.

Haven't expended any effort on decorators, good for you.

Those do sound useful, thanks for this. Will keep in mind as I continue exploring.

I think I hit a foul ball there, as run_in_executor() is for a threadpool. So yeah you are running inside of run_forever() or run_until_complete(), we just don't see it here, and that's where your sending things to run in a different thread. ;-)

Ah got it, thanks for the correction and info.

It is easy if you know what to do. But you'll CONSTANTLY run into things where it is obvious that this is a toy that hasn't been thought through.

I've found random slowdowns of a factor of 10 for no reason.

A context manager needs to have __enter__ and __exit__ methods. An async context manager needs to have __aenter__ and __aexit__ methods. The error you get if you use a context manager as an async context manager or vice versa is big and scary and offers no clue to most programmers of what the problem is.

Using async with SQLAlchemy it isn't hard to wind up exiting the event loop before all cleanup has happened. They fixed the main cause of this, but it isn't hard to still trigger it.

And so on.

Reply on news.ycombinator.com