The WebSocket Handbook
ably.com
ably.com
However the latter use offers very powerful benefits with some difficult downsides. On the positives side you get a "real time" API to work with, and you can handle events as they happen and send updates back to them. In some cases our API users can even respond to a chat message faster than we can!
Since WebSockets are virtually just a transport, it's up to you to write a protocol for handling heartbeats, authentication, and communication. In addition when you have a horizontally scaled service it can make balancing the WebSocket connections a bit more challenging since they are long lived. Deployments are even more inconvenient since (in our case) we disconnect the WebSocket consumers whenever a server is restarted for the update. It can also be difficult to fully measure and understand how many WebSocket connections you have open, and how many resources they are consuming. It's important to really push down the number of computations you are doing for users who are subscribed to the same topics so that when you send out 10,000 updates with the same message it's just the text being sent, not 10,000 DB queries :D.
Isn't avoiding this the main selling point for BEAM? As in Erlang: the movie. Can't that be done with websockets?
However, hot swapping is not super common in practice. Mainly because it's added complexity that most people can live without.
Disclaimer: we've been using Ably for years and their service and reliability has been outstanding, and we have worked closely with their engineers and I've found their expertise to be above what you may expect from experience with other company's support personnel.
Matt, co-founder of Ably
If there was a less scalable, but still mostly functioning open-source substitute for Ably, then I'd be way more comfortable using Ably in the main public cloud deployment.
> If there was a less scalable, but still mostly functioning open-source substitute for Ably, then I'd be way more comfortable using Ably in the main public cloud deployment.
Thanks for the feedback, I will pass on to the product team!
Matt, co-founder at Ably
May I suggest that the solution could be to design your WS protocol to be reconnect-friendly, like the Phoenix LiveView protocol? Maybe make a protocol which assumes and expects that a connection may be dropped by any side at any time, with robust context-restoration API such as “full snapshot” or “all updates since event ID”
The one thing i really wish websockets had is some kind of application-level acknowledgement or backpressure.
At the server end, you're blasting out messages to the client, but you have no idea if it is keeping up with them. Most of the time, it will be, but if there is a sudden spike of activity, suddenly all your dashboards are going wild, and the client may start to struggle. At that point, you want to be able to shed some load - delay messages a bit, then drop any message which gets superseded (eg if "reactor core temperature is 1050K" is buffered and you get "reactor core temperature is 1100K", you can drop the former). To do that, you need feedback about how far the client has got with processing messages.
You can build a feedback mechanism like this into your application protocol on top of websockets easily enough. But you probably want to do that from the start, or else you will, like me, one day look around and realise that retrofitting it to all your dashboards is a monumental effort.
The RSocket protocol might be a good start - it provides reactive streams semantics, and has a binding to websockets:
In one application I was streaming jpeg frames over a websocket and by the time the server application experienced backpressure there were 10s of seconds of messages buffered between the server and client. So the message rate would eventually settle into a rate the connection could sustain but messages would take 10+ seconds to reach the client.
Or, perhaps you need to tune the TCP-Window to your application.
I'm sending large messages, ~150 kibibytes, so much larger than a typical internet packet. So I'm not sure Nagle's algorithm is the problem.
> tune the TCP-Window to your application.
This is a possibility. I already had to increase wmem_max to handle fast udp connections.
I'll try to put together a minimal test case when I get the chance.
In my apps i do indeed detect the socket buffers filling, just as you suggest, but pretty much only as a way of detecting completely wedged clients.
You’ll get proper back pressure on websockets with synchronous clients that read messages actively.
This is a browser API spec problem more than a protocol problem.
I'm pretty convinced all these popular features could be layered in a reasonable way that could be implemented in most messaging systems, and have standardized semantics and conventions. It seems like every time, we're reinventing the wheel, and half the time people talk over one-another because we're using inexact language.
Basically, what I want is a "message passing a la carte" paper.
Something like a combination of Greenspun's Tenth Rule[1] and Zawinski's Law[2]. Plus whatever would include your queueing system of choice.
Though honestly I've seen more bad queues than emails or lisps. By an order of magnitude or two.
[1]: https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule [2]: https://en.wikipedia.org/wiki/Jamie_Zawinski#Zawinski's_Law
At the end of the day I think RSocket is probably the way to go for most people, though the simplicity of omnistreams is still appealing to me.
EDIT: I just learned about WebSocketStreams[2] from another comment[3] and sounds like they may solve the backpressure issue natively.
[0]: https://github.com/omnistreams/omnistreams-spec
[1]: https://iobio.io/2019/06/12/introducing-fibridge/
The WebSocketStream API is a small improvement, because it can leave backed-up messages in the socket buffer, but it still means you're depending on socket buffers for backpressure, which i think is not enough. There's still no way to actually set the receive socket buffer size in the browser, is there?
As far as I know there's no way to set buffers. IIRC there's a buffer value you can check which is what I tried to use first but don't think that got me very far. Seems like Chrome and Firefox handled it differently or something.
That means that you won't even know that a client is stuck for a minute and a half (plus however long it takes to fill your send buffer!), and even if you then throttle back, the client has a minute and a half of high-rate data to work through before it catches up. If you throttle up again once you see that the buffer is clearing, and the client gets overloaded again, you will keep hovering around that buffer full state, and the client will keep reading significantly stale data.
To get a useful real-time signal from socket buffers, you need them to be really small. But to get nice smooth transfers of bulk data, you need them to be big, so that is what is the default.
> We are generally in support of a mechanism that addresses the use cases implied by this solution document. While major questions remain open at this time -- notably, multiplexing, the API surface, and available statistics -- we think that prototyping the proposed solution as details become more firm would be worthwhile. We would like see the new WebSocketStream and WebTransport stream APIs to be developed in concert with each other, so as to share as much design as possible.
https://mozilla.github.io/standards-positions/
Unclear what the WebKit (Safari) folks think, based on https://lists.webkit.org/pipermail/webkit-dev/2021-September... that has no replies.
Microsoft is just doing whatever Chrome is doing with Edge, so I guess it'll appear there sooner or later, but can't find any public information.
Bit early to start using WebTransport seems to be the conclusion.
Matt, co-founder of Ably
It's a proposal, 2 editors are from Google the other one is from Microsoft.
I wrote the guide with example code for people wanting to know how to implement the complete WS13 protocol from scratch, so you can try it out, fiddle around and modify it to your needs.
The guide is more in-depth and assumes that the reader is willing to read the RFC when they're stuck :)
[1] https://cookie.engineer/weblog/articles/implementers-guide-t...
> willing to read the RFC when they're stuck :)
I owe RFCs for just about everything I do, they play a small but necessary role, I can't imagine not wanting to dip into one of them even if you aren't stuck.
The first websocket message is the original request, which will have the users cookies / headers where your session information / bearer token should live.
If someone ends up actually doing this in a production system, remember to not to log the accessToken if you're logging full paths/URIs somewhere, as query params usually is a part of that type of logging.
Matt, Ably co-founder
Best solution might be to generate a short-lived one-time-use ticket and pass it in the querystring.
However, websockets are not subject to the same-origin policy, so this exposes you to CSRF [1]. To protect against that, you should check the Origin header on the server side.
[1] https://christian-schneider.net/CrossSiteWebSocketHijacking....
I'm sure there are repercussions to this on the client-side, but I haven't gotten to that point yet. I'm still writing the server and testing it using automated integration tests.
My questions are:
1) Since multiple people are working together how does one manage conflicts, i.e. 2 people sending different edits simultaneously.
2) If one clients gets disconnected (4g) and then reconnects later how does it sync the changes it made during it was offline?
I recently watched this RAFT presentation (1) and I think I would need to use something like this?
What other alternatives are viable?
Also can I make it happen using just PHP, Javascript and MySQL?
Thanks
Matt, Ably co-founder
It uses operational transformations ("OT") to manage conflicts, and it saves the data in MySQL. Technically any Django DB backend will work for storage, but the public demo instance uses MySQL.
One of the reasons I made this thing was to show that realtime apps don't need to require heavy frameworks or unusual databases. And it loads super fast.
I don't think you need Raft if you have a central database storing the document. You could also consider using CRDTs instead of OT, which may be more powerful but also more challenging to develop.
CRDTs can be the answer. We actually wrote about them recently (https://ably.com/blog/crdts-distributed-data-consistency-cha...), and there is more coming soon as our Chief Of Scientist and his team are researching CRDTs and building demos!
ofcourse u will need websocket to sync changes.
I'm actually using my project to build a collaborative IDE (designer like Figma): http://www.adama-lang.org/
I'm going to be launching it as a SaaS soon so people can spin up a new back-end without managing an infrastructure.
When WebTransport reaches prime time, I'm confident we'll be supporting it.
Do people still watch football?
What are the viewership numbers for Liverpool FC?
Does Man United still matter?
:-)
I recently wrote about the Woes of Websocket: http://www.adama-lang.org/blog/woe-of-websocket with an errata based on HN feedback: http://www.adama-lang.org/blog/more-websocket-woe
The depth of this topic is very interesting, and I'm excited as I'm building some of the final pieces for my SaaS (which could compete with Ably).
HTTP servers have already solved traffic management, load balancing, scaling up and down, zero downtime deployments, A/B tests and experimentation and lots more to such a degree that we don't have to even think about them anymore. All of these problems come to the forefront again when you have to scale websocket connections beyond a single server.
When a message comes into an instance, you push it to Redis and have all of your other instances subscribed to it. Messages sync in real-time and the experience is transparent.
I teach the technique here: https://cheatcode.co/courses/how-to-implement-real-time-data...
We have a cultural challenge of how to manifest the opportunity and benefits presented by context rich communication over the entrenched ideology of statelessness and HTTP.
In our experience, many enterprise networks/vpns/firewalls still break websocket connections even when using wss, and it should not be used as the only communication channel even if you target evergreen browsers.
Out of interest, what geography and industries are you operating in where you see such a high rate of incompatibiltiy?
Matt, Ably co-founder
Implementing your own service logic is incredibly helpful in the cases where you have multiple sockets to manage and custom logic associated with identity, reestablishment, custom data handling, and so forth. There are features in the protocol that aren't used in the browser, for example, and allow for custom scaling.
Here are my learnings about web sockets:
* They are session oriented so that means both end points have to agree to connect. That mitigates many security risks associated with HTTP traffic.
* Web socket messages cannot be interleaved. In 99% of cases this isn't an issue, because control frames unrelated to a web socket message can occur anywhere without interruption. This becomes a problem if you are transfer a large file that takes a substantial amount of transfer time. All other messages must wait in a queue, which means long delayed microservice status updates or you just break things.
* Web sockets are so much faster to process than HTTP. A web socket is primitive. There is no roundtrip (request/response), no headers, and no additional negotiation. I reduced some test automation in my personal application from 45 seconds to 7 seconds by fully converting from HTTP to web sockets for messaging.
* Reliance on web sockets simplifies so much of a service oriented application. I used to rely upon callbacks to HTTP responses to verify message completion and perform next step actions in an application. Instead I am switching to specific messaging for everything. A response is a specific message when the responding machine is ready. This eliminates response timeouts, flattens the architecture, and eases service testing by moving all messaging concerns to a single listener as opposed to listening for responses versus requests from other machines.
* Since web sockets are session oriented they are potentially more fragile than HTTP. If the pipe drops you have to reestablish the connection before sending/receiving service messages.
Don't WS connections send headers? What's the limitation here?
* Doesn't natively support binary data.
* If you're using HTTP/1.1 in the browser, you'll be severely limited in the number of SSE connections you can have going at a time. If you're on HTTP/2 then it's not a problem.
In my experience, Boost Beast[1] is the easiest library to just get going with but you have to deal with all the Boost-isms that comes with. libwebsockets is the 'standard' C implementation but unless you know the websocket RFC front to back it's quite difficult to work with and has a lot of foot-guns.
[1] https://www.boost.org/doc/libs/1_78_0/libs/beast/example/web...
1) the large "implementation surface" of three components (TLS+HTTP+WebSocket), each of which by itself requires an API more complicated, if provided by a userspace library, than a kernel-provided TCP socket, and maybe
2) the fact that non-Web servers are rare enough, and WebSockets are still recent enough, that no "standard" library has emerged and had its edges honed down over time to support multiple applications, especially when the current era and funders of open source are probably less incentivized to create application-independent libraries than in earlier eras where "let's work together to create a free OS with minimal effort" was a larger share of the driving forces.
- With TLS you can certainly link with OpenSSL, but the API is more complicated than a kernel-provided TCP socket, and async/nonblocking TLS requires an API much more complicated. TLS sometimes requires a write in response to a read, and to do that in an apparently nonblocking fashion either (a) the application needs to include callsites back into library in its event loop to tell the library when the underlying socket is writeable just in case the library had something buffered it was hoping to write, (b) the library needs to run its own thread that blocks on the underlying socket, or (c) the library can only be used with languages that support async behavior in a more composable way, which is not C. None of those are good options.
- Parsing the incoming HTTP request is tricky and there's no "standard choice" for this either, e.g. a library that's been distributed in Debian/RedHat/Homebrew for >10 years and is depended-on by a bunch of applications.
- The WebSocket protocol requires that a server write a pong in response to an incoming ping. As with TLS, this means a nonblocking implementation requires a thread or integration with the application's event loop, but it's arguably even worse because WebSocket wants the server to respond soon to a ping. (By contrast, TLS-on-TCP is mostly designed so that an app can ignore the read or write direction as long as it wants.) So you don't just need to possibly queue up that pong and later call into the library when the socket becomes writeable; you need to make sure no other event is going to run or block for a long time in the meantime.
So I think the comparison here may not be, "Why isn't there a library that provides an API for WebSockets that's almost as simple as a kernel-provided TCP socket?" (where the kernel basically runs its own thread and does the async work behind the scenes), but maybe more like, "Why isn't there a user-space library that implements QUIC [or nonblocking TLS, or user-space TCP] with a simple API?"
We have implemented a nonblocking C++ WebSocket/TLS server in the cleanest fashion we could (https://github.com/stanford-stagecast/audio/tree/main/src/ht...), also for a low-latency audio project, but it's still a ton of code and has to make its own assumptions/demands on how it gets invoked. If you wanted to adopt a WebSocket implementation into Ardour, I'd be happy to help you make that happen, but it sounds like you very reasonably were looking to outsource this to a library where your application isn't the only user.
We ended up using libwebsockets and it's fine. It runs in its own thread and the rest of the code doesn't have to care much about it. It would have been nice to just use the socket directly from liblo (OSC library), but the current arrangement seems perfectly OK.
Fundamentally, there is nothing special about a WebSocket over a socket rather than a special handshake, some framing, and the layering within an existing HTTP server. The problem is that the market of developers is vastly different. If you are a systems person, then chances are good you know sockets decently. If you are a typical web-dev, then the chances are not so great and its easy to make a mess that is then exposed to the world.
I've mentored teams, and the key challenge isn't technical but education on all the gotchas.
I haven't programmed anything with websockets yet, but I read the wikipedia page about them recently and found it sufficient to understand what they were. The rest is a matter of javascript programming that I've avoided messing with so far.
I have yet to read through it much, but it is interesting.
https://files.ably.com/website/documents/ebook/the-websocket...
On the browser to server it gets a bit heavier because you need GET /path?data=<message> HTTP/1.1\r\nHost: kinda.verbose.com\r\n\r\n and then each request gets a response that can be either zero so 200 OK\r\nContent-Length: 0\r\n\r\n or contain a sync. response. It looks bad but trust me that verbosity is a rounding error when it comes to the real bottleneck which is CPU concurrent atomic parallelism, and for that you basically need to use Java:
https://github.com/tinspin/rupy/wiki (Most people disagree but the VM + GC and Javas memory model allows for atomic shared memory like none other, not even C/C++ can compete because you need a VM with GC to make that memory model work, they tried to copy it into C++11 and that was a faceplant of epic proportions that is still C++ memory model).
1 - A websocket "frame" has a variable-length header. Client->Server the header can be 6, 8 or 14 bytes. Server->Client it can be 2, 4 or 10. This is to support payloads < 125 bytes, < 2^16 and up to 2^64. I wish it was just a fixed 4-byte length.
2 - Frames can be fragmented to support streaming (where the sender or possibly a proxy doesn't know/want to buffer the entire response ahead of time). I feel like this is unnecessary in 99% of the cases. It wouldn't be so annoying..except control frames can be interspersed within fragmented frames. This is so that you can send a "ping" while streaming a large message over multiple fragments. Why didn't they just use one of those reserved bits for this?
3 - Client->Server payload is masked with 4 bytes (bitwise xor) so every message your server gets has to be unmasked.
My answer would be use HTTP/2 + SSE whenever you can get away with it. The primary limitation of SSE in this case is you can't natively send binary data (you would have to base64 encode it or something). If you're just using JSON or another text format anyway this isn't an issue.
Whether this qualifies as a 'cool' case is probably subjective, but it is important in practice.
I'm not sure if it's as widespread, but many exchanges use websockets in the front-end to make the same data available to the site users without frequent polling.
Couldn't find the Show HN just now but searching "subspace" comments might reveal it ("this reminds me of subspace"). I always wanted to look more into the approach.
My use case was I wanted to transfer a PDF to an Android-based tablet but in the moment didn't want to log into my email. I couldn't think of any quick, easy, and cross-platform solutions so I decided to write the service.
It's built on socket.io which is a godsend for websockets because it automatically handles so much grunt work and your app just works.
1. Sports events streaming live updates (see https://ausopen.com/live-scores, we are streaming live scores for the Tennis Australian Open right now). Companies like Toyota even use us to facilitate engineers tweaking the performance characteristics of their cars in realtime remotely.
2. Edtech - we have numerous customers using us to drive live classroom environments, think shared white boards, collaborative tests, presence, teacher engagement. You may have used Codewars in the past, that uses Ably under the hood for example, https://www.codewars.com/.
3. Live screen sharing and collaborative applications. You may have used the amazing Tuple.app, that uses Ably under the hood https://tuple.app/.
4. Collaborative and live web and mobile SaaS applications, where changes need to occur concurrently, notifications need to be presented, and other realtime updates are needed in the interface. You've probably heard of Hubspot, they use Ably under the hood to power their collaborative and live features, https://hubspot.com
5. Developer infrastructure and platforms that need realtime capabilities at scale. You have probably come across Split, the leading feature flag company backed by Atlassian and Microsoft. They use Ably under the hood to power billions of feature flag updates in realtime each month. https://www.split.io/
6. Financial market data - typically streaming updates to thousands or millions of subscribers with very low latency, and sometimes using features like Deltas (https://ably.com/documentation/realtime/channels/channel-par...) to keep bandwidth & latency as low as possible.
I could keep going, but I hope that gives you the idea, realtime is not just for Christmas or for Games :)
Matt, co-founder of ably.com
The client-side code is generally very simple. A dashboard opens a websocket and attaches a handler which parses the message (all our payloads are JSON), then routes the update to the right bit of the UI. We wrote a thin wrapper round the browser websocket API to handle disconnections. We wrote server-side libraries to support our patterns of use.
I initially did a bunch of developer-facing dashboards using server-sent events, because they're slightly easier to work with. However, websockets have a significant advantage over SSE: you can have a lot more of them open at once. Browsers will limit you to a few (six?) connections per origin, including long-lived SSE connections, whereas you can have dozens or hundreds of websockets open [1]. If you are serving lots of different dashboards off a single server, you rapidly get to the point where this matters!
[1] https://stackoverflow.com/questions/26003756/is-there-a-limi...
Finally, after 60 pages of docs I can Google, there’s a section called “Scaling Websockets”, which is an interesting and challenging topic.
Turns out it’s one paragraph long, saying - “Yeah it’s hard. You should consider using Ably. Next book will cover it.”
Shameful.
It's good that you are interested in Scaling WebSockets, because that is the chapter I am writing now! I hope to get it live in a couple of weeks, once it is I can send you the new version.