Asyncio, twisted, tornado, gevent walk into a bar
bitecode.dev
bitecode.dev
So twisted code can actually look like:
@inlineCallbacks
def doIt():
responseBody = yield makeRequest("GET", "/users")
returnValue(json.loads(responseBody))
Some of our initial code snippets:
# Twisted
def __protocolSendRaw(self, name, args=""): deferred = defer.Deferred() self.__EventQueue.append((name, deferred)) self.rawSend("%s %s" % (name, args)) return deferred
# Gevent
def _protocol_sendmsg(self, name, args=None, async=False): if self._closing_state: return Event() _async_res = gevent.event.AsyncResult() _uuid, event = _async_res.get() return event
> PLIVO (an abbreviation for ongoing life-threatening violence) is a procedure for cooperation between the police, the fire service, the rescue service and the healthcare system in incidents where life-threatening violence is perpetrated against several people.
Absolutely fuck yield-raise based async implementations. Just remembering those induces psychic damage; implementing async as a massive hack on top of the language is impressive but also the kind of thing we're going to whisper to junior engineers as scary campfire stories for DECADES.
async def doIt():
responseBody = await makeRequest("GET", "/users")
return json.loads(responseBody)
We use it in EdgeChains https://github.com/arakoodev/edgechains
Concurrency has a lot to do with sharing one resource, and Python has dedicated tools to deal with that depending on the resource you must share.
If you have to share one CPU while waiting on the network, then the specialized tools for this are asyncio, twisted, trio, gevent, etc.
Asyncio is the current standard to do this, but tornado, gevent and twisted solved this problem more than a decade ago. While trio and curio are showing us what the future could look like.
But chances are, you should use none of them.
So, no event-oriented programming in Python, or by some other mechanism, or…?
> If you need to get a few URLS fast, a ThreadPoolExecutor is likely the Pareto solution, 99% of the time.
I agree with this. ThreadPoolExecutor is easy to use. In general, using threads in Python to handle concurrent I/O is pretty simple and plenty performant.
asyncio with an executor is a good way to run complex tasks in parallel.
If you're a junior and have no clue, your best bet is to ask a senior in your team though.
My guess he's saying that anything that introduces concurrency is hard, and best avoided if you can do so. He's not wrong. True event driven code (without these libraries) introduces a lot of unneeded concurrency. You see it at it's worst in older UI's. There a central event loop receives mouse, keyboard and io events from the OS. The event handlers it calls transforms those events into higher level events like "field changed" and "focus change", and re-injects them into the event loop, where they get turned into still higher level events like "modal dialogue closed" until the job is done. It's a prick of a way to write a program.
If you just want parallelism (as he defines it in the article), threads with their conventional stack model are far easier to use. The problem with conventional threads is that you also get unwanted concurrency and the heisenbugs that go along with that. But green threads (aka fibres, aka cooperative multitasking) eliminate that sort of non-determinism. Green threads are what effectively gevent provides. That's why he's so positive about it.
The other libraries he discusses aren't about event oriented programming either. Rather, they tame the event loop, allowing you to write code in a way that looks similar to threaded code. And they do it without introducing unwanted non-determinism. But they come at the cost of coloured methods, so you have 2 ways of doing everything. That is why asyncio has no ftp / smtp libraries - because they have to be written in the different colour. With green threads (and gevent) all that old code continues to work. That is why these libraries suck compared to green threads. But if you have a language like javascript that doesn't have threads, they are a huge leap forward over the raw event loop processing so you'd take it any day.
What has me scratching my head is why Python introduced these higher level event libraries at all. They own the language - they could have gone the green thread route. As for Rust introducing colored code - words fail me.
Twisted, however, is a different beast. Have spent s decent chunk of time working on Matrix synapse homeserver[2], written in twisted, and oh my it just sucks.
Edit
Green threads > Green threads in other languages: https://en.wikipedia.org/wiki/Green_thread#Green_threads_in_...
Coroutine > Comparison with > Threads, Generators: https://en.wikipedia.org/wiki/Coroutine#Implementations_for_... :
> Generators, also known as semicoroutines, [8] are a subset of coroutines. Specifically, while both can yield multiple times, suspending their execution and allowing re-entry at multiple entry points, they differ in coroutines' ability to control where execution continues immediately after they yield, while generators cannot, instead transferring control back to the generator's caller.[9] That is, since generators are primarily used to simplify the writing of iterators, the yield statement in a generator does not specify a coroutine to jump to, but rather passes a value back to a parent routine.
> (However, it is still possible to implement coroutines on top of a generator facility)
Asynchronous I/O > Forms > Light-weight processes or threads: https://en.wikipedia.org/wiki/Asynchronous_I/O#Light-weight_...
Async/Await > History, Benefits and criticisms: https://en.wikipedia.org/wiki/Async/await
Another term for this asymmetric coroutine, as opposed to symmetric coroutine. Coroutines in Lua are asymmetric, but they're not generally referred to as generators as they're much more capable than what are called generators in other languages, like Python. This is largely because Lua's coroutines are stackful rather than stackless, which is an orthogonal, more pertinent dimension when implementing concurrency frameworks.
You can implement symmetric coroutines using asymmetric coroutines, and vice versa, by implementing a higher-level library that implements one using the other. So in principle they have equivalent expressive power, formally speaking. Ultimately the distinction between these terms, including generator, comes down to implementation details and your objective. And just because a language name-drops one of these terms doesn't mean what they provide will be as useful or convenient in practice as a similarly named feature in another language. Other dimensions--stack semantics, type system integration, etc--can easily prove the determining factor in how useful they are.
This paper from one of the Lua creators digs into this in the paper Revisiting Coroutines. It is a very readable paper. https://www.inf.puc-rio.br/~roberto/docs/MCC15-04.pdf
Is it necessary to use a library like trio for nonblocking io in lua, or are the stdlib methods all nonblocking with big-O complexity as URIs in the docstrings?
Trio docs > notes on async generators: https://trio.readthedocs.io/en/stable/reference-core.html#no...
[ tornado (FriendFeed, IPython Notebook (ZeroMQ (libzmq)),), Sanic (asyncio), fastapi, Django Channels (ASGI), ASGI: Asynchronous Server Gateway Interface, django-ninja, uvicorn (uvloop (libuv from nodejs)), ]
The Async/await keywords were in F# (2007), then C# (2011), Haskell (2012), ... Python (2015), and JS/ES ECMAScript (2017) FWICS from the wikipedia article.
When we talk about concurrency and parallelism, what are the different ~async patterns and language features?
Processes, Threads, "Green Threads", 'generator coroutines'
Necessarily, we attempt to define such terms but the implementations of the now more specifically-named software patterns have different interpretations of same, so what does Wikipedia have or teach on this is worth the time.
https://www.uvicorn.org/deployment/#gunicorn :
> The following will start Gunicorn with four worker processes:
gunicorn -w 4 -k uvicorn.workers.UvicornWorker
PROMPT: Generate a minimal ASGI app and run it with Gunicorn+Uvicorn. What is uvloop?
PROMPT: How does uvloop compare to libzmq and eBPF? Which asynchronous patterns do they support?
PROMPT: Which Uvicorn (security) http headers work with which k8s Ingress pod YAML attributes and kubectl?
PROMPT: (Generate an Ansible Role and Playbook to) Host an ASGI webapp with Kubernetes (k8s) Ingress maybe with ~k3d/microshift locally
I don't share any reverence for the Twisted callback patterns that AJAX also implements. And, the article isn't about callbacks.
Promises in JS have a separate success and error functions. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
MDN > Async function: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... :
> The async function declaration creates a binding of a new async function to a given name.
> The await keyword is permitted within the function body, enabling asynchronous, promise-based behavior to be written in a cleaner style and avoiding the need to explicitly configure promise chains
I wish the article had spent more time on this. Without more info I would probably just use what the author mentions at the end as the Pareto ideal solution (ThreadPoolExecutor) because usually async frameworks in not historically async languages end up being islands within the larger community that need their own bespoke libraries.
Update: I think I’m conflating a bit what I want to speed up with what is allowed. Presumably heavy compute stuff is perfectly compatible with asyncio but it won’t speed it up - it would speed up a lot of io operations. ThreadPoolExecutor can speed up heavy compute by parallelizing (if it’s heavy enough) but may be overkill for just downloading 20 web pages at once.
On top of that, you can write a C extension that will let you run compute heavy work in a way that allows your task to be suspended (for example, the C extension spins up its own thread). This extension has to be "very careful", basically by avoiding touching Python-side data during this work.
The way this sort of stuff ends up working is you pass data into a C extension, and that extension takes ownership of the data or copies it or whatever, does what it needs, then gives Python back some result.
But if you're just pure-Python compute heavy, then your task won't be suspended. So everything will run, but your compute-heavy stuff will hog the CPU, and won't be suspended. (Though if you have compute heavy work that is, like, looping over data, you could add `sleep(0)` between every couple of iterations. This gives other tasks a chance to run! This could be good enough to prevent weird bottlenecks).
But the ultimate thing is if you have N compute-heavy tasks, you probably won't get speed advantages. If you have 1 compute-heavy task and N IO-heavy tasks, you can get advantages (even if the IO-heavy stuff is interspersed). But if you have N compute-heavy tasks and not much IO-heavy tasks, multiprocessing can get you where you want (since it's usually IO-heavy stuff that is helped out the most with async/await)
But if you are trying to make a network request, disk or memory access, or any HID stuff, you might have to wait on that thing to do it's job and get back to you. At this point you can tell your thread to raise it's hand and say "hey, I don't know how long, but I know I need to wait for this thing to finish, so while I wait someone else can use the CPU, but when I am done waiting, I will need the CPU back." This is the core idea of "cooperative threads" or green threads or co-routines or any sort of thread that is at the language runtime level and not at the OS level.
https://en.wikipedia.org/wiki/Cooperative_multitasking
So consider if the task you are doing is asking for data from "somewhere external to my thread" then you might be able to make that call non-blocking to enable more throughput.
Hope that helps explain it!
for example, say you have an aiohttp-backed web server, and in the handler for some URL route, you do a CPU-bound computation that takes 5 full seconds to execute.
a crucial thing to remember is that asyncio programs, by default, are still single-threaded. so for those 5 seconds, your computation is the only thing running. other event loop tasks are unable to run. the loop is "blocked" in the same way it would be as if you called `time.sleep()` (not `await asyncio.sleep()`, but the regular synchronous sleep method) or made a blocking IO call (in a pathological case, you might open and read a file on a remote NFS share mounted over a WAN link, for example)
now, suppose your web server exposes an `/alive` healthcheck endpoint. during that 5-second interval where you're hogging both the CPU and the event loop, aiohttp won't be able to dispatch requests to the healthcheck endpoint. if your load balancer has a 3-second timeout for those healthchecks, from its perspective the service will appear to be flapping between online and offline, even though the service itself never actually goes down.
the "polite" thing to do, for this sort of big CPU-bound task, is to hand them off to a ThreadPoolExecutor like you said (or ProcessPoolExecutor if you have GIL concerns) with `loop.run_in_executor` [0]. that gives you a future, and when you `await` the future you are politely yielding the event loop to allow other tasks to run, such as those requests to the healthcheck endpoint.
0: https://docs.python.org/3/library/asyncio-eventloop.html#asy...
It doesn't speed up the I/O operation, but allows the CPU to continue executing other code rather than blocking and waiting (and the program becoming unresponsive).
If your app is querying a database ten times per user request and averaging 5-10ms per query, your CPU is spending 50-100ms doing nothing but waiting on the network to finish so it can resume executing the code that comes next. That's time other code that has CPU instructions that can execute now would benefit from.
https://github.com/python-greenlet/greenlet/blob/master/src/...
I still think it's too much for a Python project -- if you think you need this in 2023, maybe you need to reevaluate? I'd seriously have another look at Go.
Before anyone jumps in to say nobody ever has/will ever have to troubleshoot this because it's so stable and used everywhere and so forth, please have a look at the comments at the top of that file. This code had to be patched eg. due to a compiler update.
Remember, this is open source and this code is offered without any guarantees. And non-existing guarantees of greenlet/gevent is quite different from the non-existing guarantees of CPython (no idea about PyPy).
Implementing cooperative multitasking at this level is way off the beaten path and should only be attempted by those who know what they are doing.
Today, I'm sure this code works perfectly on linux 6.4.7 when compiled by gcc 12.3.1 against python 3.10 on amd64 architecture in release mode. But can you guarantee it will it work on linux 7.8.17 when compiled by gcc 15.2 against python 3.21 on AARCH64 10.4 architecture? No you can't. If you are lucky, it'll just die with a general protection fault or something. But it can also break in subtle ways that only a seasoned systems programmer can realize, let alone fix.
So if you are a professional, you care about maintainability, you care about forward compatibility, you are starting a greenfield project, you are sure you need cooperative multitasking and you ask nicely my opinion about which async framework you should choose, I would strongly advise you to steer clear of gevent/greenlet based libraries.
Or, you know, just hack away :)
The article provided a grocery list of concurrency primitives which IMO: was designed to paint the misleading picture that concurrency in Python is too hard (when everyone just uses the excellent and standard asyncio library in practice.) Then the article ends with the faulty conclusion not to use any concurrency primitive. (Again, why? Where is the logic in this conclusion?) Python's asyncio library handles both CPU-bound work and I/O bound work. It has data structures for queues, call backs, push and pull style usage, sleep, 'background' execution, future results, and much more. It's very well designed and tested.
The article vaguely alludes to async tasks being hard to debug because their execution order is obscured. But... that's kind of the point of using them. Programs written to use this logic flow don't have to be weighed down waiting for every result. It's a huge benefit in Python land where you have a GIL on a core and it's able to switch between multiple tasks without blocking the others for no reason. The article didn't provide any reason why you would want to consider other options to asyncio which makes their inclusion seem like more of a flex than anything.
I'm wondering now if most of the commenters here were just OPs friends shilling their support for something they don't understand.
Anyway, in 2023, if I need something that rely heavily on async I/O, I will use go without any doubts. One of my side projects is basically a scraper: I started from normal Python, then multithread, then asyncio, then trio and at the end I ported to go+colly with a lot of benefits (speeds and easily of maintenance).
I remember using Twisted some 10 years ago. Never again...
surprised the author ended on the note that none of these are relevant - i use a lot of these tools on a weekly basis.
This also answers something I've always wondered about which is how Twisted and Tornado fit into the Python web framework landscape and whether I should use them. Tornado always seemed popular but slow and less intuitive than Flask/Django. And then Twisted was a far lower level library but people were still building APIs on it.
Where does Eventlet fit into the picture? Is it a similar box of magic monkey patches like GEvent?
Not true. Future executors can run in processors pool (with different cores) or using threads (same core, different event loops.)
>You have to understand that async programming is hard, and no matter how good the tooling is, it's going to make your code more difficult to manage. It has a high price.
Dude starts out with an article that lists the entire kitchen sink for concurrency; Neglecting to mention that literally none of these libraries are even considered by Python devs. Then concludes that writing async code is too hard (writing 'await' before a call is too hard.)
To me this entire article reads like someone's attempt to over-complicate things by trying to overwhelm you with information to seem impressive even though 99% of it is irrelevant to the discussion. Then makes a non-sequitur conclusion to make it seem like only 'experts' ought to touch any of this because [for you] (le simple-minded pleb) you will inevitably break everything. Lmao, fuck that, that's condescending as hell. Literally just using async def for functions that do IO and write await. There's a few things to learn but any dev is capable of doing it.
>I’ll add you probably should not go with asyncio manually. Use a higher level asyncio based lib, or better, framework. Async is hard enough as it is.
Horrible advice. Asyncio is high level already and comes standard in Python 3.6 >=. You will get far more trouble using third-party concurrency libraries than trying to use the Python library which is actually very elegant and well-supported. Twisted sucks, by the way.
If only it were that simple.
> You will get far more trouble using third-party concurrency libraries than trying to use the Python library which is actually very elegant
Again, you overlook the fact that asyncio requires you to explicitly launch a concurrent context. Asyncio is fine, but it is not "elegant" because of this.
`asyncio.run()` [0] was added in 3.7, and as far as I know is the current recommended way to do this.
as shown in the asyncio "hello world" example [1], you can write your main() function to be async, and then the entire process runs asynchronously from the beginning.
and as shown on that page, you can also use `python -m asyncio` to get a REPL with a running event loop where using `await` will work as you'd expect it to.
one of my biggest quibbles with a lot of asyncio demo code snippets is the use of `loop.run_until_complete()` [2] and similar lower-level primitives. in most asyncio applications you simply want to start the loop as early as possible with `asyncio.run(main())`, let it run for the duration, and have the entire process exist in async-land.
0: https://docs.python.org/3/library/asyncio-runner.html#asynci...
1: https://docs.python.org/3/library/asyncio.html
2: https://stackoverflow.com/questions/40143289/why-do-most-asy...
I don't think there's any possible way to do that elegantly.
maybe gevent-style monkeypatching makes it look elegant, but there's a whole pile of inelegant stuff happening under the covers to present that illusion.
and I would say that almost by definition, if you're monkeypatching the stdlib, you are banished from the realm of elegance.
Is there a practical distinction? It feels like there isn’t. It’s not as if a two CPU machine can run two Python processes “in parallel” as opposed to “concurrently”. I have 100 other processes running too. It’s all concurrency, and never parallel.
Perhaps the author and others are distinguishing between concurrency that can happen at the granularity of any instruction (e.g. threading) compared to cooperative yielding that only happens when await is called (e.g. asyncio.)
I’m not sure if this will help, but I think of it like - parallelism is running the same task in multiple threads / on multiple cores at once. And concurrency is “concurrently” running different tasks on one core.
Nodejs is a purely single threaded runtime. Let’s say you have a web server in node processing http requests.
- Your program implements concurrency by yielding back to the event loop instead of blocking. Request 1 comes in, and then sends a query to the database. While we wait for the database to respond to the query, we can process request 2.
- You usually implement parallelism with nodejs by running multiple instances of your nodejs server process across all your cores. (Then use a program like nginx to load balance requests across all the node processes.)
The GPU might be a better example. When rendering a frame, the graphics card runs a small program (called a fragment shader) for every pixel on the screen. A 4090 has ~16000 cores which can all run the same fragment shader on different pixels at the same time. This is called parallelism, not concurrency. Concurrency would be if you had two video games open at once and your graphics card was quickly swapping between rendering frames for game A and game B.
If they're both CPU-bound they probably can. Most of the time those 100 other processes are probably sleeping and not actually using the CPU (unless you're at 100% CPU usage all the time? My computer tends to sit at around 5-15% usage)
Gevent takes over, often making advanced asyncio not work on gevent programs. Patching things globally should obviously be seen as dangerous in itself.
I feel like car sharing services are a better analogy. If you're not using the car, someone else can.
That being said, I disagree with this: "I strongly believe beginners should start their first serious project with django and not flask, despite the fact most people see it the other way around"
Given the dearth of django content for beginners, flask seems much easier to learn. This is based on my own work over the last year, so YMMV.
Is it unfashionable to say that NodeJS got this right with > v7.6? It's always felt pretty straightforward to me but perhaps it just suits my mindset. It's only when I try to do this in other languages (e.g. Python or Ruby) that it starts to feel clunky.
“We're having some trouble serving your request. Sorry!”
What I really like about asyncio is that there isn't much of deps or forced loop in order to get it working.
But I could be mislead and just not know how it is achieved in tornado, does gEvent burn the codebase as well ?
I still miss Perspective Broker and Producers/Consumers. Those things were rather powerful.