Hacker News Reader

Comment by strictfp | Hacker News Reader

I find it amusing that TCP was designed as and still is a full-duplex protocol for long-lived connections (streams). Then someone decides to use it in HTTP (1.0) for short-lived, half-duplex, message-oriented communication. Not surprisingly, it is not a good fit. Fast-forward a couple of years and everybody is busy de-crippling TCP in HTTP with various hacks trying to delay the HTTP request-response cycle, thus uncovering the streams in the underused underlying protocol. First now, many years later, the mistake is properly corrected and TCP on the www is vindicated through websockets. Funny world.

View on HN

Wait a little longer and, if we are very lucky, multicast will be "re-discovered" as well. That way, instead of having servers shove data down many sockets to many clients, they will write their payload down 1 multicast socket to all interested clients, with said payload crossing the network as little and as far as necessary but not more.

So goes the wheel of progress...

Multicast is a nightmare over the general internet, from what I hear. It'd need some significant tooling to work properly.

Sounds like ZeroMQ's "rediscovered" multicast as you describe.

"not a good fit"

You're speaking of HTTP: the protocol on which a great deal of world's computing infrastructure is being set on top of. The protocol which we are communicating with each other over right now. HTTP on TCP is extremely successful and achieves all it was designed to for and has been extensible to handle much more. There are many basic problems which are abstracted well into stateless request response cycles: file serving, RPC. There was no "mistake" - it was not a bad fit. There are only expanding use-cases involving existing software infrastructure - for which websockets is solution. That's not to say that HTTP is perfect... It could be better in many ways - but it's ridiculous to write off its success and call it a mistake.

I am not dismissing HTTP. The mistake I refered to was to limit TCP usage for www by only allowing the request-response oriented HTTP protocol. HTTP is opinionated and its REST-based architecture was explicitly chosen to disallow partial updates of web pages, thusly crippling TCP. But if one wants to build "thick client" type apps the restrictions of HTTP will weigh you down hard. Allowing for unrestricted TCP usage makes a lot more sense in that case, and is exactly what WebSockets delivers.

One can argue that we were better off with the pure REST web before XmlHttpRequest, in fact I am inclined to agree there, but the crippling of TCP was bound to be cracked. The real humor is that the crack is now re-launched as a new great feature, when it was explicitly excluded for the break-down of REST that it causes.

You say HTTP crippled TCP so as to not break-down REST. HTTP 1.0 does not seem to be designed for REST; it doesn't seem that REST was really published until quite a while later. Calling HTTP "REST-based" seems a bit of a stretch.

On top of that, could you give a few examples of what you actually mean? Like what would a partial update be?

TCP seems like a perfect fit for the underlying transport for a request/response model; I don't see how choosing it is some sort of deliberate "crippling". What should they have done? Built a custom request/response protocol on top of IP? Isn't that just crippling IP's flexibility as a layer 3 protocol? I don't understand your objections.

Roy Fielding was involved in HTTP 1.0 and I believe that REST was a generalization of some of the principles used therein. I explained the breakdown in another comment, see http://news.ycombinator.com/item?id=4033717

I'm not saying that TCP was a super-bad choice, just that they wanted only a subset of the features and got a bit more than they wanted. Also see http://news.ycombinator.com/item?id=4033822 .

Supply & demand.

Back in the day internet connections were so terrible that it was unbearable to do much more than to download a document and display it. Hence, HTTP and HTML. Arbitrary TCP streams existed but in practice performance and latency were very limiting, so nobody cared that HTTP had no support for streaming.

Eventually the underlying tech matured and the standards followed suit. Slowly. But that's the price of interoperability.

I too find it amusing how these concepts keep getting "rediscovered", but in hindsight it's not surprising.

So what underlying protocol would you have used instead?

Back then TCP was the best mainstream choice. Using UDP would have looked good due to its message orientation. But its limited message size, lack of re-ordering and congestion control would have disqualified it almost directly. TCP then looks better, since it has reliability, re-ordering and rate control. You only need to change the streaming functionality for senfing messages. This they did by using one TCP connection per message. What they missed, however, was that connections are expensive in TCP. There are a lot of handshakes and TCP probes a lot before opening the throttle, something you notice when downloading a big file. Throwing away connections like that is a waste. Ideally, you would want one connection per server and reuse it for multiple requests. This was done in HTTP 2.0, where connection:keep-alive was introduced. But TCP still has significant overhead and a slow rate control mechanism. It also does not allow multiple concurrent requests in the same connection. These problems are addressed by SCTP, which is really a great fit for HTTP and awesome in general. It is also message oriented. SPDY is another similar alternative, which is less general than SCTP.

But the point I was trying to make is that TCP was intentionally crippled by HTTP to acheive REST. Then people hack the restrictions of TCP and declare a new invention.

Reply on news.ycombinator.com