Hacker News Reader

Comment by lmilcin | Hacker News Reader

To be fair, this can be fixed with well designed libraries that don't rely on TCP doing job for them merging buffers and preventing small writes.

The issue is vast majority of libraries treat the problem as if it did not exist and prefer to not get their hands dirty and just conveniently write a stream of data to the socket leaving to the user to correctly configure options on the socket.

But yes, in general, performance is at least in significant part about remembering a huge amount of trivia.

View on HN

Well, yes. The point of TCP is that it's an opaque reliable linear-stream abstraction. If you're not treating it like an opaque reliable linear-stream abstraction, you shouldn't be using TCP. If you want to manage your own datagrams, use a datagram transport. (Not necessarily UDP. I'd suggest SCTP, personally. Or maybe QUIC.)

That's naive at best. You say, if I wanted to write good HTTP client I should decide not to use TCP because TCP is meant to relieve the user from managing datagrams?

The reality is, when you design a client library you work with what you have and in case when you are writing HTTP client what you have is TCP on one side and a user application on the other side that expects maximum performance and minimum latency possible (just look at the benchmarks on the net that compare tens of frameworks in competition for who can produces largest TPS of hello world with lowest latency).

HTTP is specifically the case where Nagle will probably hit most often.

If you treat TCP just like file stream (it's a stream after all) and you're implementing, say, a webserver, the most straightforward way to implement a response is to:

  - figure out what was requested
  - build headers
  - send headers
  - send the file body

especially if you intend to send the file body using sendfile. but this pattern is broken by Nagle - because there's no return traffic between the two sends in HTTP and the headers often won't fill out a full TCP packet, you'll trigger the wait.

The solution I have implemented in my case is to move the problem to the application by designing the writes a little bit better. Instead of writing into dumb stream the application writes buffers of data or stream of data but each time indicating if there is more data that will immediately follow.

In any case, TCP_NODELAY is enabled and all writes are immediately flushed unless the application clearly indicates it has a queue of data that will immediately follow.

In your example sending the file body will fill the remainder of the packet containing the headers and trigger a packet send. The client waiting for your response isn't going to lose out. In fact, they'll benefit from having file body data in that first packet.

Your example is when you want Nagles algorithm to apply. This exact use case is mentioned on tcp(7) under the TCP_CORK option, which is like a more explicit, application controlled, version of Nagle.

Nagles algorithm sucks for HTTP clients who happen to send big HTTP requests. If the HTTP headers being sent are >1400 bytes or so, then the tail end of the request will wait 1 RTT for the ACK of the first part. This will be particularly bad if the HTTP server has delayed-ACKs enabled (TCP_QUICKACK is not on). The server TCP stack has no idea that the HTTP server application hasn't yet got a full request and can't yet respond, so will delay the ACK of the first 1400 request bytes in the hope that it will.

The delayed ACK delay is probably small compared to the RTT, so Nagle still dominates

True, but also kind of irrelevant, in that HTTP1.x is a protocol designed under the constraint that there are going to be an unbounded number of application-level proxies/gateways between the client and the server, where some of them might be under the client’s control (e.g. Tor), some under the server’s (e.g. Varnish), and some neither (e.g. a corporate or ISP WAF.)

And because of that, modern web servers don’t really do NODELAY; rather the opposite — they fully buffer HTTP requests and responses into what are essentially single giant datagrams.

Put your Express app behind Nginx, and Nginx will buffer the entire request to feed it to your app as one TCP jumbo-frame (if possible); and then buffer your response to feed it back down the line to the client as one TCP jumbo-frame, or a burst of such packets. This happens not just with HEAD responses, but with full-bodied GETs/POSTs/PUTs, too.

The only time modern web servers attempt to break your buffer up and schedule sends at different times, is when you tell them that you’re doing a `Transfer-Encoding: Chunked` response; or when your upstream is HTTP/2 and sends down an HTTP/2 stream non-terminal body-frame; or when your upstream is speaking websockets or WebRTC. Otherwise you get this “hyper-Nagling.”

(Why do they do it? Because these servers want to achieve sub-worker-thread concurrency, and system calls like write(2) are blocking, getting in the way of that worker doing anything else. So web-server worker threads want to do as few system calls as possible, which means batching up your writes so they can do one context-switch for one big write(2) or writev(2). For these servers, sendfile(2) can actually be an anti-pattern, compared to mmap(2)ing the relevant static files so that they can just be passed as a pointer to writev(2) along with the headers buffer. That’s one blocking context-switch rather than two!)

Your suggestion makes sense when you control both end-points.

If you only control one end-point, you might be stuck with TCP. See also the write-ups around BBR, like https://blog.apnic.net/2017/05/09/bbr-new-kid-tcp-block/

For BBR, it's about server-side changes to TCP with unmodified clients.

If your library is already merging buffers and preventing small writes then you would still want to set TCP_NODELAY, to eliminate delays due to send/send/recv patterns where merged buffers are less than the MSS... because you know for sure that you're already doing all you can, and Nagle's algorithm can't help further except introduce delay.

That's exactly what I meant. Did I say otherwise?

The general approach is to disable the algorithm (ie. enable TCP_NODELAY) and then implement whatever optimizations you can on higher level.

Reply on news.ycombinator.com