Async Views in Django 3.1
testdriven.io
testdriven.io
C# probably has some of the best async primitives of any language. I still managed to be bitten by the very common issue of error logging getting lost in a hidden callback that doesn't look like a callback because of the magic of the async keyword (a common issue in node as well).
I'm convinced that the async style of web servers is largely a step in the wrong direction. A leaky abstraction that we don't need.
A better model already exists - thread per connection - for the majority of what we want as web developers. There are some good things about allowing us to manage our own waits but those things are rare.
It reminds me of the choice of when to manage your own memory - usually we don't need to, but sometimes we do. Most languages programmers use today decided, rightly, that memory management was an exception and not the norm. We are more productive today for the reduction in cognitive load for most applications as we decide not to manage our own memory.
Thread per connection has gotten a bad name for performance which is pretty false. You can create the exact same underlying abstraction with green threads for more concurrency when you absolutely need it (but mostly you don't). The code we get from not using async is reduced in complexity substantially.
I'm disappointed that django is on the async bandwagon. It means that I may have to make another framework choice in the future.
I suspect the actual motivation is that async is virtually required to address performance problems associated with the new tag helpers - which are themselves a terrible, terrible idea. Slow, brittle, surprising, wholly reliant on syntax highlighting, and then redundant because you can implement the same functionality using the existing C# syntax.
What do you mean, any example? I have code full of asyncs and never had problem with logging
AFAIK, only Go does this well (with great results) as a result of it being pretty much baked into the language. I don't think it's an async bandwagon, I think async/await is the best way to get concurrency without requiring the whole language to opt-in (as the Rust devs discovered).
But I agree with your main point, concurrency is hard if the language isn't built from the ground to be concurrent.
* Different OS' handle things differently. macOS limits the number of threads per process (somewhere around 1500 I believe). Other OS' may be slower, or impose other limits
* Because of this, sometimes spawning threads is pretty fast (e.g. 10 µsec). Other times it's super slow (I've seen threads take over 500 ms to spawn)
* Context switches are still pretty slow across OS'. There's finally some work being done towards lightweight thread support in Linux, but I suspect it will take a few years to become useful
I _really_ hope that one day we _can_ "just" spawn 100 000 OS threads without issues, as it would make many concurrency problems easier to solve. Sadly, I think it will take another 10-15 years.
The other big problem on Linux is stack sizes - "hundreds" or "thousands" of threads isn't a problem - the C10K problem which proved the dominance of nginx's evented model vs. Apache thread-per-conncetion model was broken 20 years ago. Where it matters is when we reach 100s of thousands or millions of connections on a single node. The first issue you run into is the thread stack size - Linux has a hard default of 8MB; at 100,000 you start to hit OOM issues first. Go (and BEAM) has a ton of work to make this use case possible within the runtime.
What I'm unsure about if Linux even cares about solving this problem - the threading model is pretty general and I'm not sure how important high performance connection handling is. At the end of the day, if you are really I/O bound, an async style, co-operative, event queue will always be more performant than a generic threaded scheduler like the one Linux has; meaning if you care about performance (like Django probably does) they would probably go with an async style scheduler even if Linux could support a million threads.
This is not how thread stack sizes work. Thread stack sizes allocate a certain amount of _virtual_ memory. This means that with a stack size of 8MB you don't actually immediately use 8MB of physical memory. Only when you start filling up that memory do you start using physical memory. We can easily demonstrate this using a simple Rust program:
use std::thread;
use std::time::Duration;
fn main() {
let mut handles = Vec::with_capacity(10_000);
for _ in 0..handles.capacity() {
let handle = thread::spawn(|| thread::sleep(Duration::from_secs(1)));
handles.push(handle);
}
for handle in handles {
handle.join().unwrap();
}
}
/usr/bin/time -f '%M KB RSS' program-name
88492 KB RSS
The virtual memory limits in turn are not much of an issue. I think on most 64 bits systems the virtual memory limit is 256 TB. A 256 TB limit would allow up to 33 554 432 OS threads with a stack size of 8MB. And since you can configure the limit when spawning threads, you can increase this number by spawning threads with a smaller limit (e.g. 4MB).
They also read from disk for most requests. So, thread per request and also reading from disk.
I worked at one company where we had many dozens of services with thrift/spring boot and spawned a request per thread and it worked fairly well. We were collecting and aggregating every review on the internet (for our customers), and had dozens of services... There were a couple in a reactive style. So we took the maintenance cost when we needed to.
Async code has its traps, but they are very managable once you've learned them. I haven't found them on the level of manual memory management which is a footgun that even experienced devs keep firing.
The benefits are also there, doing slow things in parallel like HTTP calls or database queries is trivial with async code and trickier in sync code.
Of course you prefer sync code when you don't need it, but async/await style code doesn't prevent sync code. If you only support sync doing anything async because very cumbersome and dangerous, and doing so requires things like managing threads or locks yourself which is an even greater recipe for bugs.
You don't have to use locks either - straight parallelism can be done very easily with the Executors in concurrent.futures. There are a lot of good concurrency primitives in the threading library in the stdlib including conditions, semaphores and barriers. Of course best to avoid parallelism with shared data if at all possible (and IMO it doesn't really come up much).
I see many issues from developers where I have to walk them back to the start and spend time on identifying the problem. There are many reasons for this, but it's not necessarily the most fun or exciting thing for developers to do. There are still many artificial impediments because devs don't universally have the control over, or access to prod environments that startups may have. If this troubleshooting is a hurdle then sprinkling in some async starts to seem much simpler.
Honestly, in a lot of cases when I have thrown away something and used something else because I just need to show that I'm doing something to develop a reasonably performant application rather than spending all my time optimizing performance. Spending a whole week optimizing your application's performance is surprisingly rare in the modern hustle-driven culture of "lean" startups and such -- cargo-culted by a lot of companies and bosses who have nothing to do with startups.
I recently realized that the entire gunicorn / uwsgi world for example is filled with anecdotes about performance instead of simple, comprehensible benchmarks; and I thought they were one of the better ones. I suspect this kind of lack of performance focus by most of the users is a big part of why.
Its hard to create a simple benchmark for something that has 100+ configuration options and can be run in dozens of fundamentally different ways.
C beats Python though by a large margin.
If you watch Andrew's talk from PyCon AU last week he explicitly calls out that sync views are still better for most cases, which is why he invested so much effort in making sure they were not negatively affected by the addition of async: https://youtu.be/ibAmA4QQDhs
For the API gateway use case this feature is really useful, and extends the usefulness of Django by letting it serve as a gateway for longer as your app grows and you start extracting services from the monolith.
I think part of it is the idea that in principle calling your DB is IO - but in practice your DB is fast enough that your sync view is not spending much of its time waiting on that IO.
Note that simonw is a Django core dev, so when they explain what the design goals were for this feature you can be confident they are correct.
I’ll just quote it here for clarity:
> The async design for Django is predicated on the idea that the vast majority of cases are better handled by threads - but very occasionally it is useful to run an async view
This is an empirical question though, and will vary depending on workload; if you are doing something funky with long-running stored procedure calls, then you might get a win with async (make sure your DB driver is async too though, the default ones in Django aren’t last I checked). Again, my claim is just that most use cases are not like that.
I suspect its because the event loop gets blocked doing trivial things like json/csv parsing, but it could just be python's slowness.
Asyncio seems super hard to profile too..
For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here.
Native threads are huge improvement over async, but still not as good as green threads, which really are the best option here for 99% of usecases, abstracting whole async thing away.
> Thread per connection has gotten a bad name for performance which is pretty false.
With 64 or more cores being the norm, a thread with 2MB stack per connection requires 128MB just to exist (though I believe OS may offer some magick if you are not actually using it). This is probably 3-4x time what's required for green threads.
Sounds good to me!
Can you even find a VM offering that has a 64 core CPU and less than 32GB of RAM?
> For that you will need a language with good support for threads, which Python definitely isn't (at least not CPython), so Django doesn't have any other option here
The thread per connection is managed by your application server. Why does the language need to have good thread primitives then? In fact I would argue you spawning and managing your own threads make your service less robust.
Because: a) we want application server to be written in the language we are using, so that we can control every aspect of how it works, instead of having some silo that developer will avoid because its so different b) those threads will share cache, connections, some preloaded data you may need and so on, you want a language that allows this to be done reliablyl c) those threads may need to create more threads (make parallel requests to 3 services and resize an image at the same time)
Are you assuming that each thread requires a dedicated stack on each core? That isn't how it works - each thread has a single stack regardless of the number of CPU cores. So, a 2MB stack requires just 2MB. However, that 2MB is 2MB of virtual memory. So, assuming that the stack isn't allocated using a 2MB huge page, it will probably actually be physically allocated in 4K chunks as required. So, most threads are going to use considerably less than 2MB of actual memory. Outside of 32bit systems, virtual memory is close to an unlimited resource.
Green threads also require stacks. How green threads allocate stack space is going to depend on the particular runtime, but it's likely that like with real threads they will also lazily allocate memory as required. A big difference between stack usage with green threads and regular threads is that the runtime _may_ support shrinking the stack used by a green thread if space is only briefly required by a green thread, while a regular thread will hold on to any memory that is physically allocated for its stack until it exits. Reclaiming memory like this isn't free, however - and you run the risk of "stack thrashing" where the point where a stack grows and shrinks is accidentally in the middle of a hot loop and you end up spending basically all your time growing and shrinking the stack as opposed to doing real work.
I _think_ that some green thread runtimes may also support stacks smaller than 4K - but I'm not really sure if / how many actually do.
No, I am assuming we run 1 thread per core.
Haven't we already seen in Apache vs Nginx that thread per connection will OOM the web servers under heavy traffic? (haven't used Apache in many years so this may be obsolete)
Although it could be argued that fibers are a type of primitive thread--so as long you don't mean OS thread the idea has merit.
[1] http://httpd.apache.org/dev/whiteboard/process-model.html
Anyway, I agree that async seems to be more trouble than it's worth for most cases.
The async afficianados also bring along the async ecosystem split.
I wish they did a better job of bridging this gap.
Languages like rust, which already have amazing support for threads don't need this split, please.
It was created by the same guy who wrote the excellent & highly popular Django-Rest-Framework. Very lightweight, clean, well-documented, and a breeze to get running with. It also supports async tasks natively.
I spent years working with Django and now I'm currently using it to power the api for https://sqwok.im.
Our cms, is based on Guillotina (https://github.com/plone/guillotina) that uses a Datastore more or less equivalent to the Zodb, and plone, but internally uses postgresql and asyncpg. We are active contributors on it, and since latest version it works around asgi (uvicorn). We also have some plans to make it a bit more psqlidiomatic (get rid of pickles, ....)
On other internal services, we use the raw asyncpg.pool, mostly because part of our bussiness logic it's inside prostgresql, and this doesn't feet super well with an ORM. Think on a 15 year old sql schema, that had evolved through different hands, and right now it's multitenant xD
Latest opensourced thing, it's a work queue based on plgsql, with asyncpg bindings. (https://github.com/vinissimus/jobs/).
So, we can say, we use asynpg in all possible forms.
It's neat, it's a query builder, not an ORM or any nonsense, and it makes queries a lot more manageable (bringing SQL logic into python primitives).
Can anyone who has worked in both types of applications comment on their differences?
- wholly implicitly (by monkeypatching stuff with gevent typically)
- async/await
- by reactor (twisted)
async/await is not really an alternative to the explicit style of twisted. To paraphase Guy Steele it is an attempt to drag the monkeypatchers halfway to twisted. It doesn't really solve the problem of mixing sync code in with your async code, the only preventative to that is programmer diligence. If you read from a file using open in the middle of your async code you're going to block the process while bytes are gathered for you. However it does prevent you from accidentally using async code in the middle of your sync code.
Problems like backpressure and failure handling are largely unsolved in asyncio.
I like to compare it to Typescript/Kotlin-style null-safety v.s. the Java Optional or Scala Option. If you want make code null safe with option you have to rewrite basically your complete code from "procedural" to chains of (flat)maps. In contrast in Typescript you just change T to T? in a few places and add an if where you want to handle the optionality.
async/await versus reactive streams has this same problem, reactive streams is highly impactful, much more than async/await. In practice I've seen this have the effect of turning applications turn into a small reactive stream outer shell with the app itself being mostly synchronouse/Future, while async/await is used a lot more liberally because the impact is smaller.
> It seems like a worse abstraction because it mixes asynchronous code with synchronous code
It doesn't, at least in Kotlin only an async function can call an async function. This isn't different from using `map` in a stream.
'Behaviors' in FRP help a bit here... but if you can avoid the complexity, why not?
For when we do FRP, I like how we got it in our React code: Mostly synchronous functional code (React::render(), ...), and then we do animations and other stream-like concepts via Rx. Our nodejs code works similarly: mostly async/await route handlers, and now and then we do fancier http streams and whatnot via Rx. (Though even there async/await has been increasingly nice, as we generally aren't doing many fancy temporal operators nor contention handling.) We haven't figured out an equivalent solution for our Python async code, however as async/await is still new & changing, and afaict FRP/Rx never really made it there.
Citation needed... It will be slower by definition but it probably advertised higher throughput anyway. I wonder how many people the throughput question is true for. You’d have to be pretty hardware constrained.
They even suggest asyncio could be a replacement for celery... It really is the Wild West out here.
Asynchronous code is absolutely not just for running tasks after returning an HTTP response. A good example of the true power of asynchronous execution would be to do a Web Socket handler _inside a Django view._
Only if you don't care about reliability.
Yeah this is pretty questionable.
Having state in your app's process and then having to pretty much re-invent queues, job cancellation, uniqueness guarantees, scheduled jobs, periodic jobs on an interval, queue draining and retries with exponential backoffs.
I mean, yeah you could code all of that with enough time but why would you when Celery exists. If anything having the queue's state held in Redis or anywhere outside of your app's process is enough of a reason to pretty much never replace Celery with asyncio. Even in the post's example of sending an email I would still want a lot of what Celery has to offer and wouldn't consider replacing it.
IMO Celery isn't going anywhere soon and I would still use it in every Python project that needs background processing.
Yes that is complete madness. The last thing I would ever want to do is send an email or use an external API (those are the examples in TFA) in an async thread as part of a request-response cycle. How does error handling or retries work for that background stuff? I don't know the exact ins-and-outs of Djangos new async stuff yet but my guess is that if that mail/API server doesn't respond to your IO then your response to your own HTTP request hangs?
Above all the central and undemonstrated claim is that there are any performance or management advantages to async. I tested sync vs async on this a couple of months ago. The best performance and robustness is with garden variety sync python with a good WSGI server:
http://calpaterson.com/async-python-is-not-faster.html
If anyone has any contradictory benchmarks please let me know. I haven't been made aware of any yet.
Async results in better utilization of system resources... since there aren't entire threads with app state blocked on IO requests (which would consume a chunk of system memory.)
Considering that a web app is most commonly waiting for a database operation, a single server can serve many more active requests simultaneously (since each active request is not allocating an app instance.)
The trouble with that benchmark is that (and they're fairly open about this) they aren't keeping things the same between frameworks - there are multiple variables. If you look at the Flask code all the queries are being put through SQLAlchemy but for various async frameworks they're using the database driver directly and using native code libraries for JSON decoding.
I think techempower's benchmarks are interesting but they don't help you do a comparision of async vs sync.
Additionally on the particular benchmark you've linked to #1 is a dead experimental framework that is not in wide use and #2 is Falcon, a sync framework that is in wide use.
I worked past 2 years with asyncio and I have to say it still kinda sucks. The syntax is overly verbose and asyncio itself can't decide what it wants to be. However it still works _good_ for the most part and there's an enormous community and effort in this!
however, is it stable enough and roboust to replace sync methods in production for the benefit of these cases?
That's why some years ago I gave up on Django altogether and moved to Nodejs. Much better experience writing async code.