Comment by zaphoyd | Hacker News Reader

Comment by zaphoyd | Hacker News Reader

I think his analysis is flawed. WebSocket is a message based protocol that does not specify a maximum message size in the RFC. This does not make it a streaming protocol until an implementation decides to deliver incomplete messages to the end application. Some implementations have done this, many (including all browsers) have not and will not.

Time and time again it has been demonstrated that we are bad at choosing a maximum allowed value for all applications and all future considerations (see: ethernet frame sizes, IP address lengths, operating system address spaces, file system block sizes/counts, etc).

In some cases (many of those previously listed) there were hardware, cost, or technical concerns that led to nailing down a number in an RFC. For WebSocket there is no clear benefit to forever encoding a specific numeric maximum message size. It is a high enough level protocol that there is no technical or cost benefit to make message sizes limited by anything other than individual application needs.

As such, the WebSocket RFC leaves maximum message size implementation defined, and specifically says that an implementation SHOULD implement a reasonable maximum message size for its purpose. A chat application that knows it will only be moving small text messages can set its maximum message threshold small to improve buffer performance and catch invalid messages sooner. An application that finds a business case for sending a large file in one large message can set itself up accordingly. Generic WebSocket parsers should expose a method of setting the maximum message size the application wishes to receive.

I definitely agree that not requiring implementations to return their maximum message size along with the "Message too big" error will make some sorts of interoperability more difficult. However, it also prevents exposing implementation security details and simplifies the core spec (the author has already complained that the spec is too complicated already). It is relatively simple for an application to negotiate a maximum message size privately if necessary and the WebSocket extension mechanism allows a method for standardizing a way of doing so if this turns out to be a serious issue in the future.

I've no problem with the lack of a max message size in the RFC, what could cause problems is the fact that it needs to be passed between client and server "out of band", i.e. at the application protocol layer rather than at the websocket protocol layer. Also bear in mind that this blog entry was written based on Draft HyBi 09 and not the final RFC; the wording has changed somewhat since then.

The draft in question suggested that providing a message based interface to application code was possible and that the parser could/should deliver only complete messages to the application code. That's hard to do if you also want to allow for the 'endless streaming' scenario that others on the working group were fond of. The result was a bit of a mess.

The final RFC addresses some of this, but there's no getting around the fact that the websocket protocol itself can't tell you how big a message is until you get the final frame.

Sure you can work around all of this even for a generic parser but the initial wording in the draft in question could lead you towards the wrong design if you're not careful.

As I mentioned above, I agree that that needing to pass maximum message size out of band makes some things more difficult. Whether that was the right tradeoff in terms of convenience vs protocol complexity I think has yet to be seen. At any rate, an extension to perform this in band should be trivial. Perhaps I will try writing one to test out my extension handling code.

I do see your point on the "endless streaming" section of the RFC. Stating that "(section 5.4) The primary purpose of fragmentation is to allow sending a message that is of unknown size when the message is started without having to buffer that message." implies that a web socket implementation should support this sort of operation. Indeed, if you want to support sending messages of unknown size you must expose an interface more complicated than the default message based one.

That said, a message only implementation that does not allow sending unknown sized messages is 100% compliant with both the spec and receiving such messages. The RFC probably could have made this fact more clear. I believe that endless streaming mode will not be a common use case and have not implemented it in my generic WebSocket library. I do believe, however, that fragmentation of messages provides important benefits even without unknown size sends. Once you have message fragmentation there is no additional protocol cost to allow unknown size sends.

I agree with all you're saying.

The wording of the RFC has improved since that draft and the flexibility could be useful in some scenarios.

I ended up with an API which can be asked to deliver complete messages 'if possible' given the buffers provided by the client of the API. If it's not possible and the buffer becomes full then the API simply gives you the fragment of data and tells you if it knows how much more there is to come or not.

Reply on news.ycombinator.com