Writing a chat application in Django 4.2 using async StreamingHttpResponse
valberg.dk
valberg.dk
One thing to note, it's been possible to build these sort of things with Django by using Gevent and Gunicorn for over a decade, and it works well.
In many ways I wish Gevent had been adopted by Python rather than AsyncIO.
Also, what was the reason not to go with Gevent?
The biggest downside of Gevent IMO is that it enables the magic of turning sync code into async code via monkey patching things like the socket lib. This lack of explicitness can make things a bit difficult to reason about, without a good mental model of what Gevent is doing underneath the hood.
I have 7 YoE with Django. Its great at so many things. You see some code, like middlewares, and immediately understand what's going on.
Now, we also have Starlette. The base of all new, fancy asgi libraries. Here's the base middleware class.
https://github.com/encode/starlette/blob/8d7a1cacfb3e1a30cbb...
In the last couple of years I heard 'we're running fastapi on production. Wanna join us?' so many times... but the reality is that it's still not suitable for prod. Who wants to work with a code like that if you have a readable, stable Django? I'm clueless.
I have zero trust in this code and I have been bitten by fixes to this library that introduced deadlocks in my own code.
[1] https://github.com/django/asgiref#synchronous-code--threads.
[2] https://github.com/django/asgiref/blob/main/asgiref/sync.py#...
Is there anything else in Python-land you would go with for the use case you have in mind? Or does pretty much all the async stuff turn on you like this, at some point or another?
- pick a mature ASGI server
- pick a framework and libraries that have been designed from the start with async in mind. Be suspicious of libraries that support sync and async APIs without clear guard rails between the too implementations
- avoid setups where you have multiple event loops in a single process.
[0] http://www.pgbouncer.org/features.html#fn:2I'm using internal pools + pgbouncer but as I write this out I am beginning to wonder if that is even necessary.
[1] https://stackoverflow.com/questions/10587660/how-does-jetty-...
This is mostly true today as well, although we do have beefier machines and more efficient thread pooling.
I did some personal research for infrequent real-time notifications. My conclusion was that many stateful connections are poorly memory-optimized by language runtimes and reverse proxies. Even with lightweight tech like Golang, gRPC, nginx, etc, it’s hard to push anything less than 30 kB for an idle conn, mostly from thread/goroutine stacks, user space buffers and (easy to overlook) a bunch of HTTP headers and TLS handshake state that often remain after establishing the conn. That’s without any middleware or business logic wants their share of the cake.
The only mature project I found that really took this stuff seriously is uWebSockets. It’s extremely lightweight, around an OOM better than most alternatives. Highly recommend – they also have a socket library so you can implement other protocols if needed.
Anyway, it’s important to be aware that massive amounts of long running connections is not just about adding a websocket/SSE library. Chances are you’ll need a mostly-separate serving stack for that purpose.
To those who have been around longer than me: isn’t this just long polling, that predates websocket?
The actual biggest pain for me has been the front end handling of it with the fetch api. But that’s likely just due my inexperience with it.