Comment by marshray | Hacker News Reader

Comment by marshray | Hacker News Reader

Parent Full thread

It's only capable of being used for messages if there's something to guarantee that all hops along the way are going to preserve the message boundaries in ways expected by the application layer. Can the protocol split single messages? Can the protocol merge adjacent messages without reordering?

Unless the protocol specifically guarantees certain behavior and commonly-used systems regularly exercise this guarantee, it's just not going to work reliably when it's needed.

Hearing some of the "works for me" discussion from developers suggests that we're heading for that magic situation where it works 99.9% of the time. I.e., the system looks fine in testing and then fails in mysterious ways (that require deep protocol fixes) in production.

Ideally, implementations of such a protocol would intentionally fragment the messages somewhat if they were not going to guarantee they were atomic. But there are very few developers (and code reviewing managers) enlightened enough to let that kind of thing ship.

The protocol preserves message boundaries but not fragment boundaries. You may send a message of, say, 100 bytes and get 100 x 1 byte fragments arrive, or you may send 100 x 1 byte fragments and get 100 bytes in a single frame. The main issue, for me, at the time, was that when you get that first 1 byte frame there's no way to know how big the resulting message will be.

Luckily there's a rather excellent compliance test suite, here: http://www.tavendo.de/autobahn/testsuite.html which should go a long way to help nail interop issues.

when you get that first 1 byte frame there's no way to know how big the resulting message will be

So that's a design tradeoff. I've implemented protocols that did it both ways and it's definitely easier on the guy trying to implement a library or other receiving application if he can get a reasonable upper limit on the size of the messages.

But on the other hand, by not requiring the total message length be known in advance, it eases the logical (and memory) burden on the sender. Often the sender will be an overloaded server.

Nothing can prevent a higher-level protocol on top of WS from negotiating its own max message size.

So the design choice that was made would seem to allow optimizations for the overloaded server case without prohibiting other optimizations. This is typical of W3C protocols.

Well, all frames actually do have length in the header. It's just not the first byte... it's the lowest 7 bits of the second byte, and possibly more (check the spec).

So, if I can read 2 bytes from my buffer, I can get a good idea of the message size. If I only have 1 byte, I rewind and wait until I receive another.

But we're talking about messages, not frames. http://tools.ietf.org/html/rfc6455 "frames have no semantic meaning"

If Websocket developers begin making unwarranted assumptions about message framing and fragmentation you will regret it. BELIEVE ME.

/me gets back to bugfixing

The WebSocket protocol guarantees that all hops along the way preserve message boundaries except in the case of explicitly negotiated extensions that dictate otherwise.

Even in cases where your WebSocket implementation exposes incomplete message fragments, the boundaries of the message are still clearly and accurately preserved.

Don't worry, its not as bad as all that. WebSockets doesn't actually break message boundaries, it just might fragment a message without telling you the full size up front. The basic "problem" is that if you receive a 1kb fragment with no size indication, and you expose an API that is entirely based on messages and events, then you have to start buffering. But what if the message is a 5gb .iso file of your favorite DVD? Well, there's no way to predict that so you might have to dynamically allocate a monstrous buffer, which is just painful.

There's two ways to respond to this: 1) claim that websockets is broken and all client APIs should be streaming or 2) not unexpectedly send multi-gigabyte messages.

You mean: 2) Trust everyone on the internet to not unexpectedly send your server multi-gigabyte messages

Reply on news.ycombinator.com