Server-sent events, WebSockets, and HTTP
mnot.net
mnot.net
You need to implement a server-side heartbeat feature.
You need to handle the close event from EventSource and be able to reconnect.
Tabs can be problematic. When you subscribe, you use a URL with a nominal ID to identify the client. For example, on a chat app, you would use /api/sse/userA/subscribe
Problem is, if userA starts opening tabs, each tab creates a new subscription for userA so you need to randomize each connection (userA-UUID).
If you don't use a nominal id, the server won't know to which subscriber to send the data and you don't want to broadcast all your chats.
I've used the Broadcast channel API in conjunction with SSE to have only one tab handle the SSE connection, and broadcast incoming SSEs to the other tabs which also reduces the number of connections to the server to one.
On the server it's also a PITA because not all instances/pods have the subscribers list. The way I've found to solve this is with clustering the instances with Hazelcast or Redis or a MQ.
But once you figure out all this, SSE works quite well.
It looks pretty clever, I wish more websocket frameworks like [2] supported this out of the box (with failovers for safari).
[1] https://dev.to/ayushgp/scaling-websocket-connections-using-s... [2] https://www.npmjs.com/package/hydrated-ws
https://stackoverflow.com/questions/28310501/why-did-safari-...
To add to your list of problems, I have had memory leaks with SSE responses stuck open on the server even when the client disconnects. Resorted to killing the response on the server every couple of minutes and relying on the client reconnecting.
Why don't you want to send the event to all the tabs? Why do you want to differentiate the tabs (by UUID)?
Only thing is the message ids reset when this happens.
https://html.spec.whatwg.org/multipage/server-sent-events.ht...
Granted the benefits of sse: no additional port to open, no dependencies - just slip your logic into your routing at /events. Of course this may not be important for some. Websockets are a good choice as well. I tend to id clients on ip address not connections however, depends on the specific app.
We use it exclusively for the soon to be released 1.0 version of Zotonic. See: https://test.zotonic.com (running on a 4.99 euro Hetzner vps).
We developed an independent support javascript library called Cotonic (https://cotonic.org) to handle pub/sub via MQTT. This library can also connect to other compliant MQTT brokers. Because MQTT is fairly simple protocol, it is fairly easy to integrate in existing frameworks. Here is an example chat application (https://cotonic.org/examples/chat) which uses the open Eclipse broker.
Strong disagree. MQTT has no place in a world with WebSockets. MQTT knowledge is so esoteric in comparison. Maybe it's the Haskell of transport protocols: really friggin smart, but not if you're trying to be useful to society at large.
Open pages in a browser are not that different from IoT devices.
Because despite being a dinosaur, it still only has middling adoption. It exists in a particular niche of a particular niche of computing.
HTTP, in contrast, is English to MQTT's German. One is just infinitely more common and accessible to the majority of the world. So, if you're serious, use it.
> Because WebSockets is effectively a blank canvas, there are a lot of choices to be made when designing a protocol on top of it, and no one way of doing it has yet gained momentum.
It immediately reminded me of Flash.
Flash was basically a blank canvas where you can do just about anything. Turns out that's not a very good model for a number of reasons -- one of which being it turns everything you do into a snowflake and you need to invent your own patterns and abstractions instead of leaning on an extremely well thought out but more limited approach that's been vetted for many years before Flash was a thing.
I don't think WebSockets are really bad but it makes me internally wince whenever I see tech stacks trying to push using them for everything, even for transitioning between pages with no broadcast mechanism. Now I see part of the allure though. If you're a language or framework maker you get to make decisions at the protocol level instead of sticking to decades of standards which is the awesomeness of HTTP.
I'm all for sprinkling in tiny bits of WebSockets when it makes sense, ie. doing actual pub/sub things like showing notifications or showing messages like "4 new people commented on your post, click here to show them". In the same way that it felt ok to use a little bit of Flash back in the day for specific functionality instead of throwing out everything and trying to make your entire site in Flash.
yet we're in 2022 and the only things that are remotely competitive with what Flash allowed 15 years ago, are some games put in WASM blobs (which will "work" until $BROWSER deprecates some API in two years).
I don’t think of this as a bad thing. There was exactly a lot of what you say happening around that time, but the time when Flash was biggest was also a time of massive creativeness on the web. I think we’re missing some of that now, and I’m more than ok with our collective “pendulum” swinging back towards the blank canvas than the overfit tech world we have today.
WebSockets shouldn’t be approached as a “from scratch” communication method unless you have very specific needs or your application is dead simple. Everyone else should take advantage of the numberous libraries available to help with the basics.
> I don't think WebSockets are really bad but it makes me internally wince whenever I see tech stacks trying to push using them for everything, even for transitioning between pages with no broadcast mechanism.
I’m not up to date with the less popular web frameworks. Which frameworks do that?
One of them is https://github.com/phoenixframework/phoenix_live_view which is a component of Phoenix / Elixir.
It will render your initial page over HTTP but then when you transition pages or perform actions using various functions it provides you it will send a diff of what's changed over WebSockets. These actions could be limited to only the 1 user using the site, such as clicking different links in a nav bar to transition between page A and B.
The alternative to this is doing what other frameworks like Rails has done with Hotwire Turbo where they make these types of transitions or actions over HTTP. Turbolinks started to do this back in 2015 or whenever it came out and now more recently with Hotwire Turbo there's ways to only update a tiny area of the page (again over HTTP) but it also optionally supports using WebSockets for when you want to broadcast things to all connected clients, such as updating a counter somewhere or maybe showing a new reply on a blog post, etc..
Laravel has Live Wire and there's also HTMX which is back-end agnostic which all use HTTP for a bulk of their behaviors to send server rendered HTML over the wire. I'm not a Laravel developer but based on their docs they have a drop in solution to handle broadcasting events with WebSockets on demand too[0] so it can continue using HTTP for everything except for when you want to broadcast events similar to Hotwire Turbo in that regard.
I ask as someone who's very interested from a distance in what Phoenix is doing with websockets, but hasn't gotten into it enough to understand the cons.
The BEAM and Erlang/OTP introduce an entirely different set of considerations when it comes to maintaining and managing on the order of millions of concurrent stateful websockets connections.
The architecture of Plug/ Phoenix also means that, should the community come to a conclusion that websockets are no longer right, the transport mechanism is extensible, but the underlying interface and semantics of your application would be essentially unchanged. The Elixir forums would be a better place to inquire about websockets vs. SSE implementations for LiveView.
I do understand but I don't think it's practical to use WebSockets to transport a page diff to move from page A to B, or to update a little area of a page that's not broadcast to anyone else.
The case to use WebSockets from Elixir has always been "but think of what you don't need to send over the wire like HTTP headers!"... until you want to know things like what the user's agent is, IP address and a few other common pieces of information that's normally stored in an HTTP header. Now you need to explicitly add these to your socket. It's easy (1 line of config) but these bytes persist on the socket for every open connection.
Now it means if you have your million concurrent users you have to make sure you have 1GB of memory on your server that's dedicated to storing nothing but this information. I'm being really optimistic here and only accounting for 1kb of data per connected user. Realistically a lot more memory will be used with a million users.
In the grand scheme of things 1GB isn't a lot of memory, especially if you're talking about a million users but this is a problem HTTP doesn't have. This information isn't persisted on a socket and stored in memory on your server. It's part of the headers and that's it, your server is done knowing about or caring about it once the response is sent. If you had a million concurrent visitors on your site your server wouldn't store anything on a socket because no socket exists.
Likewise as soon as you start using `assigns` with LiveView, anything you store on the socket is going to actively take up memory on your server. Yes I know about `temporary_assigns` but if you start using that everywhere then you lose out on the "surgical diffs" and you're back to transferring a bunch of stuff over the wire because the state isn't saved on the socket for each connected user.
The system contradicts itself and you end up needing a real lot of memory on your server to hold this state or you trade that off for storing only the essentials and it's back to sending a lot over the wire. This also has a bunch of mental complexity because you as the end user who builds the app needs to think about these things all the time.
It's much more than "think" too. You end up in a world where you can go out of memory and your server will crash unless you carefully pre-estimate / provision resources based on a specific traffic load to know how much memory it will take. You can go out of memory with a lot less than a million connections too, especially if you forget to make something a temporary assigns which the compiler won't help you with. With HTTP can throw a cache in front of things and now you're bound by how fast your web server can write out responses. HTTP also has decades of support around efficiently scaling, serving, compressing, CDNs, etc..
Then on top of that, if a user loads your site and there's a 3 second blip in the network after it's loaded then they're going to see loading bars because the WebSockets connection was dropped and the socket needs to reconnect. With HTTP this isn't a problem because once you load the page the response is done and that's it. If there's a blip in the network while reading a blog post that's already loaded then it doesn't matter because the content has already been served.
On paper being instantly aware of a dropped connection sounds amazing, but in practice it creates a poor user experience for anything that's read-only such as reading a post on HN, or a GitHub issue, or a blog post or just about anything that's not related to you actively submitting a form. The world is fully of spotty internet connections and HTTP is a master of hiding these blips for a lot of cases.
You can build pretty interactive sites without WebSockets too. If you bring up the network inspector on Google Documents there's no WebSockets connection open. I don't know what Google is doing here but I notice a lot of big sites don't use WebSockets.
For example AWS's web console has a bunch of spots where you can get updates but there's no WebSockets connection open. The same goes for GMail, GitHub (real time issue comments) and others.
That's not to say WebSockets are all around bad, it's just interesting that even for something as collaborative and real-time as Google Docs it can be done without them while having an excellent user experience.
NOTE: I would never solely base a tech decision on what these companies are using for their tech but it's at least interesting they've all chosen not to use WebSockets for whatever reasons they had.
> The architecture of Plug/ Phoenix also means that, should the community come to a conclusion that WebSockets are no longer right, the transport mechanism is extensible
What would happen if the transport layer went back to HTTP? Right now with LiveView you have to rewrite your entire front-end to not depend on plugs / controllers and now you have to re-create this idea of a plug-like system in WebSockets (such as using on_mount, etc.). All of this new code and patterns to solve the same things that were solved for decades with HTTP -- or let's say ~7 years with Phoenix pre-LiveView.
Those are only a few things related to the problems you face when using WebSockets for everything. I still think WebSockets are good but personally I wouldn't use a framework that pushes to use it for everything, the world isn't connected over a local connection with 1ms of latency and 0% packet loss. Plus, WebSockets powered sites tend to feel pretty sluggish for me. It's like browsers are less optimized to deal with painting changes emit by a socket, I don't have a scientific measurement to show you here but I can feel it on sites that use WebSockets to handle things like page navigation. There was a site I saw like 8 months ago where it showed both a Hotwire Turbo and Stimulus Reflex (WebSockets) demo page hosted on the same server to do pagination on a table and clicking around the WebSockets version felt slower even with the same latency to the server.
Speaking of "feel", I made a video about how Topbar from Phoenix makes page transitions with a fast connection feel slower a few weeks ago https://nickjanetakis.com/blog/customizing-topbar-to-add-a-d.... This is unrelated to WebSockets but I'm just saying it's pretty easy to see inefficiencies around user experience in ways that aren't mathematically proven.
I've never found pub/sub quite the right abstraction, because almost every implementation I've seen has race conditions or issues on reconnect. Usually its possible to lose messages during reconnection, and there's often other issues too. I usually want an event queue abstraction, not pub/sub.
I met mnot a few years ago when we (the braid group) took a stab at writing a spec for HTTP based streaming updates[1]. Our proposal is to do state syncronization around a shared objects (the URL). Each event on the channel is (explicitly or implicitly) an update for some resource. So:
- The document (resource) has a version header (ETag?).
- Each event sent on the stream is a patch which changes the version from X to Y. Patches can either be incremental ("insert A at position 5") or if its small, just contain a new copy of the document.
- You can reconnect at any time, and specify a known version of the document. The server can bring you up to date by sending the patches you missed. If the server doesn't store historical changes, it should be able to just send you a fresh copy of the document. After that, the subscription should drip feed events as they come in.
One nice thing about this is that CDNs can do highly efficient fan-out of changes. A cache could also use braid to subscribe to resources which change a lot on the origin server, in order to keep the cached values hot.
[1] https://datatracker.ietf.org/doc/html/draft-toomim-httpbis-b...
We've done some revisions since then but have been working on getting more running code before pushing forward more with the approach. Most recent draft & issues: https://github.com/braid-org/braid-spec/blob/master/draft-to...
My experience is building an exceptionally large pub/sub service, and pub/sub starts nice until you really care about reliability. I made the mistake of patenting a protocol to sit on top of WebSocket/SSE/MQTT as the ecosystem was a giant mess: https://uspto.report/patent/grant/11,228,626
What I learned was that by having E2E anti-entropy protocol is that it's hard for pub/sub abstractions to not lose stuff or lie. That protocol emerged precisely because years of investment in pub/sub was a leaky bucket of issues.
Braid looks interesting. I'm using JSON marge as my algebraic operator since I have an ordered stream of updates for my system. http://www.adama-lang.org/blog/json-on-the-brain
I was absolutely gobsmacked reading Kafka's API design, because kafka avoids this problem with a single, simple API choice.
In Kafka, instead of a client subscribing indefinitely, clients can only request ranges of events. Like, a client query will say "Get me the next ~1mb of events from ID 123". The kafka server will send those events (from disk, or streaming) then stop sending them to that client. Once the server has sent the quota 1mb of events, it'll stop and the client is then expected to request the next 1mb of events.
This simple API almost entirely prevents buffer bloat. (Can you see it?). If there's not enough bandwidth for the events to arrive, at most 1mb of events will ever be buffered for that client. Because the client will wait to ask for the next 1mb of events until after the first events have been received, if a client processes events too slowly (or doesn't have the bandwidth to keep up), it'll just gracefully fall further and further behind, without consuming any extra server resources. It'll keep working - just work slowly. And, sure - this is still a problem. But its graceful. And it entirely fits with what people expect to happen when a client doesn't have enough bandwidth or CPU. And its a much smaller problem than if the entire server failed due to a single slow client. Its genius.
It can work, and the data is at least limited by the amount and speed of producers. But it can also get a bottlneck.
> How do you even solve it, in principle for systems like this? What should happen if a client can't process all messages? Do you drop some? Should the server persist messages on disk? (And how / when?)
This will all pretty much depend on the problem domain. If one can't drop messages, then it pretty much all doesn't help. But usually in systems clients do not need all the past history - they might just need the newest data. So it's possible to detect in various parts of the system (even the server or broker before the TCP stack) that some data is superseeded with a newer version, and then drop the old variant instead of sending it too.
If purely "latest state" has to be transferred to a client, the max amount of data which needs to be queued is always sizeof(state). now this gets again tricky, because the state could be big and completely resending it might also be expensive. So a compromise solution that is sometimes used is sending the client one snapshat of the complete state when connecting, and then falling back to sending events/diffs. If one client falls behind on processing diffs, the will likely reconnect and get a new starting point. Obviously also has its gotchas, since you don't want to get to a point where there's constant fetching of initial state which is most costly. But at least it limits the amount of data that is queued anywhere in the system to "whatever is required between 2 state snapshots".
https://lists.w3.org/Archives/Public/ietf-http-wg/2022JanMar...
What it doesn’t have is clustering and a father that’s != null at marketing =]
The solution uses mqtt. Clients subscribe to a topic on the server, and the server publishes patches to update the view. Patches can be incremental (patch against the last frame), cumulative (patch agains the last keyframe) or a new keyframe. It allows for server side rendered views. Multiple clients can subscribe to the same view and keep in sync. See: https://github.com/mmzeeman/zotonic_mod_teleview
CRDTs remove the need keeping track of all (or any just recent) revisions on the server as you would with OTs, your server can be stateless and just act as a message broker between clients.
Edit:
Should have followed your links, yes that’s exactly what you are doing.
They are quite different from SSE and WebSockets in some aspects. Using Web Workers for some core feature shouldn't be considered lightly. But the end result is pub/sub, so it's worth to check out.
https://developer.mozilla.org/en-US/docs/Web/API/Push_API/Be...
The big advantage of SSE when doing a simple event stream from the server is that they can be implemented in stacks that don’t support Websockes, such as many of the older Python frameworks (i.e. Django) built on top is WSGI. To use Websockes with these frameworks they either need upgrading to ASGI (or equivalent) or you have to run an additional server process for Websockes.
I do a lot of Django, SSE are super easy with it, just a StreamingResponce implementing the SSE protocol. I have then tended to use Gevent with Gunicorn to handle the long streaming responses rather than the default threaded workers.
There are few caveats around implementing a heartbeat event and ensuring they are closed properly when the client disconnects. This can be super annoying, I tent to kill the connection every couple of minutes as you don’t always know when the client has disconnected. Also buffering in reverse proxy’s need to be worked around.
To be honest though as Django gains better support for Websockes I would probably use them over SSE even for a single direction event stream, less edge cases to think about.
For long running responses like websockets and SSE you really need to use none threaded worker processes (one websocket connection would clog up a a single worker thread). Until recently the best way to do this was with Gevent with Gunicorn. The asyncio stuff that is coming to Django is a great addition, they have just merged async querysets. However it’s not the full framework yet and so although views and querysets are async, the db adapters are not yet and use a thread pool internally. I’m hoping they get to that level of the framework soon, it’s been a sessions undertaking to asyncify the Django api.
I like the idea of continuing to use the original sync api everywhere except on websocket/SSE views or views with a lot of io (many db query’s and http api calls) where it has a proper advantage.
I haven’t looked in detail recently but I would still consider whether going Gevent/Gunicorn (which is basically magic) is better than asyncio at the moment for Django. May be worth doing some benchmarking.
I kind of wish Gevent had won the battle with Asyncio.
Not sure how often this is done, but I've been sending query parameters along with the request and using it to stream results back to the client for low-latency initial responses to long-running API calls.
My company (Fanout) is attempting to solve this via the Generic Realtime Intermediary Protocol: https://pushpin.org/docs/protocols/grip/
The idea is to enable origin servers to delegate connection management to a proxy tier, without introducing a new client-facing protocol.
What would you recommend? Websockets? MQTT?
Frameworks like JustPy have used it to couple the frontend to the backend and have a Python server doing all the logic work (i.e. frontend just passes everything over WebSockets to the server and asked what to do). It's an awesome demonstration of what WebSockets can do.
MQTT has no place on the web. It barely has a place in IoT and should lose all rights to that claim yesterday.
[1] https://datatracker.ietf.org/doc/html/draft-nottingham-http-...
To be completely honest, when you look at the big picture, HTML was designed to be a static document format.
I really wish some new format or standard would be built from the ground up. HTML JS was never a good format, it just got popular to fight microsoft.
Meanwhile Chrome is saying people don't use Push. Yeah, well, because ye jerknuts have diluted & made unusable the best part of it: the ability to be responsive to resources coming at us. What a sad truckload of tragedy this undelivered capability has been. To invent a new HTTP, HTTP2, with new capabilities, then spend most of a decade ignoring & denying developers access to the best parts of what you just invented. Irony & tragedy. Now they're taking away PUSH[3], having never made it usable at all. Classic frigging Google, what frigging fickle monsters, incapable of even the most basic follow-through on what they start.
[1] https://datatracker.ietf.org/doc/html/rfc8030
[2] https://github.com/whatwg/fetch/issues/51
[3] https://groups.google.com/a/chromium.org/g/blink-dev/c/K3rYL...
This happens to use HTTP/2 PUSH_PROMISE frames, and it’s a good use of them. But that doesn’t imply anything about exposing HTTP/2 Server Push to arbitrary web content.
(I have no idea of the implementation status of RFC 8030, whether any or all user agents have replaced their own ad hoc systems with it.)
The fact of the matter is that HTTP/2 Server Push really just didn’t pan out for general web content: its original intended purpose flopped, turning out to cause more trouble than it solves; cache digests could make this probably not so, but they make it even more complex, and add a little overhead, so that based on what’s been seen so far, implementer consensus (consensus, not just Google) is that it’s just not worth trying. Sad but true. As for the remaining possible cases, they’re served about as well and more compatibly by older techniques. (And compatibility is important: you emphatically cannot depend on HTTP/2 working, so you mustn’t require HTTP/2 features, so there’s not a great deal of purpose in implementing them at all.) So in the end, exposing HTTP/2 Server Push to client code is probably a net loss, significant complexity spent for significant risks, insignificant uptake, and negligible concrete benefits. Google is bad, sure, but I don’t think this is an example of that.
Noteably none of the alternatives deal directly in HTTP resources. HTTP PUSH could have/should have allowed sending new http resources directly. HTTP could have become a way to allow a page to be reactive, as new content was streamed at it, and this would have been a near ideal fit architecturally, building on what the web is and what http is, to meet with modern apps which have incoming new data streamed to them all the time.
Instead we have this long list of un-HTTP non-resourceful hacks, to have a separate eventing system. Because HTTP2 built the tech then failed to let the page use it. Technology like RFC 8030 demonstrate what that better future look like & is widely used, just, as you say, not by the page directly. This is a fuck up, a huge missed opportunity. I disagree with your assessment that backwards compatibility means we should stick to the same ratty old non-resourceful options we've been stuck with: we should & ought to use better. That good tech was impossible to use directly by the page was a sin that prevented this technology from receiving adoption & attention.
It feels like everyone is fixated on the cache-digest problem, which indeed is hard, but it's just not relevant. There's so many other interesting & good architectural options that this tech opened up. Just, the web never empowered developers to use them.
Just my personal preference. Not claiming anything beyond that.
HTTP was never designed for pub/sub... How does the additional layer of complexity provided by HTTP for file transfers (e.g. request headers with each request, response headers, cookies sent in each request, mime types, etc...) benefit us for the pub/sub use case? It just adds overhead and makes it harder for vendors to fully implement HTTP. What used to be a simple protocol is becoming prohibitively complicated.
I agree with this sentiment. But mnot spells out well in his article why adding it to HTTP would be pragmatic (CDNs, standardised, caching)
I was considering Websockets or MQTT (or both), so server-sent events sound like a direct and possibly superior competitor. But from a design standpoint, what do you do? Just do a GET on some general "status" endpoint to open this SSE stream and then have the server send everything down this pipe indefinitely?
While this SSE channel stays open, the controlling app will need to be able to continue to do additional traditional GET, POST, etc. calls to the server. How are responses to those distinguished from data coming in from SSE?
> Each HTTP connection maps to one underlying transport connection.
-- https://httpwg.org/http-core/draft-ietf-httpbis-messaging-la...
TCP uses "port numbers" to identify different connections. 2 different connections from your PC to the same web server will use 2 different "source" ports. A port is just a number in the TCP header. https://en.wikipedia.org/wiki/Transmission_Control_Protocol#...
HTTP/1.1 added pipelining, so you can make several requests on the same TCP connection before receiving a response. But the (complete) responses must arrive in the same order of the requests so it doesn't work for SSE.
HTTP/2 added request & response multiplexing on the same TCP connection. But (according to the OP) there are some limitations that affect SSE.
https://developer.mozilla.org/en-US/docs/Web/API/Server-sent...
This works like SSE, but is designed specifically to articulate changes to the state of HTTP/REST resources.
If you're in Javascript, you can use the braidify library: https://www.npmjs.com/package/braidify
But none of this complexity is necessary.
Most of the world's "cloud" needs could be handled by each extended family of 100-200 people have a couple nerdy cousins administering a backed-up Nextcloud instance (maybe Sandstorm or Cloudron if you really want to go nuts).
> What's the best way to do pub/sub on the Web?
I think it's the wrong question. Or at least it will become the wrong question at some point. Should we be pushing for web technologies even when we need communication models that the Web clearly was not designed for? After having used NATS for some things, web services, web sockets and all other related stuff feels archaic and byzantine. Too much ducktape on top of a very simple, but limited idea of hyperlinked documents.