Hacker News Reader

Comment by tptacek | Hacker News Reader

This isn't a defect, which makes the whole comment kind of strange. I blame the post title, which should be "Golang disables Nagle's Tinygram Algorithm By Default"; then we could just debate Nagle vs. Delayed ACK, which would be 100x more interesting than subthreads like this.

View on HN

Certainly you'd agree that this is a bug in git lfs though, correct? And users doing "git push" with their 500MB files shouldn't have to think about tinygrams or delayed ack?

It's reasonable to think about what other programs might have been affected by this default choice (I'm sure I used one myself two weeks ago—a Dropbox API client with inexplicably awful throughput) and what a better API design that could have avoided this problems might look like

I don't know enough about git-lfs to say. Things that need buffering should deliberately buffer, I guess?

Maybe golang should default to panicking if the application repeatedly calls send() with tiny amounts of data :)

Ok, I've replaced the title with that. Thanks!

though I kind of liked "This adventure starts with git-lfs" (the old use-first-sentence-as-title trick) which was the replacement before this

I think it's a false dichotomy. Delayed ACK and Nagle's algorithm each improve the network in different ways, but Nagle's specifically allows applications to be written without knowledge of the underlying network socket's characteristics.

But there's another way, a third path not taken: Nagle's algorithm plus a syscall (such as fsync()) to immediately clear the buffer.

I believe virtually all web applications - and RPC frameworks - would benefit from this over setting TCP_NODELAY.

It would also be more elegant than TCP_CORK, which has a tremendous pitfall: failing to uncork can result in never sending the last packet. And it's easy to implement by adding a syscall at the end of each request and response. Applications almost always know when they're done writing to a stream.

Why isn't this a defect? It brought OP's transfer speed over Ethernet to 2.5MB/s.

It's an option, with a default. Arguably (I mean, I'd argue it, other reasonable people would disagree), Go's default is the right one for most circumstances. That's not a "defect"; it's a design decision people disagree with.

you clearly (in this post and yours others) did not read OP and other comments on this thread where it's documented that it WAS NOT design decision. why use it as an argument where it's written it was NOT by design.

the same with LFS -> this post clearly shows detriment to LFS usage, and probably many other tools written with golang.

'most circumstances': prove it, or dont use.

If there is a defect, it's in git-lfs. Picking a reasonable default is not a defect.

It being reasonable is what's in dispute.

Not really, not on this thread. The debate is valid (though maybe not in this hyperbolic framing), but this is subthread where I'm responding to someone who "picked their jaw up off the floor" at this "defect" of a very obvious default in the Go standard library that has been there I think since its inception, as if no network software in the history of software had ever deliberately disabled Nagle, rather than that being literally standard socket programming advice for decades.

(Again, being standard advice doesn't make it not debatable!)

I. Don't at all understand this comment. You don't own this subthread? I don't really even really recognize the boundaries of subthreads from the larger thread, at least not in the way you're suggesting? The article is about surprising consequences of this decision. This being a "good default" is very much a subject of contention in this discussion.

> (Again, being standard advice doesn't make it not debatable!)

This seems to accept my premise that it's what's in dispute?

I don't claim to own the thread, but since you've jumped in to respond on behalf of the other person I responded to, I'm going to to tell you again that what you want to talk about here isn't what I'm here to talk about. There are plenty of other subthreads here talking about whether disabling Nagle by default is a good thing or not; maybe join one of them.

I'm not responding on anyone's behalf. I think your attitude here is really weird. You are in fact asserting that you are the arbiter of what can be discussed in this subthread. If you don't want to discuss what I'm discussing - just don't respond? Telling me to go away is so strangely aggressive, I'm baffled.

I'm not going to respond any further because this seems very unproductive.

Isn't that just how threads (and debates in general) work?

I even tried emailing tptacek to try and be part of the change he said he wanted to see. Crickets.

HN folk can be a bit hypersensitive and / or opaque at times. Text medium is not always ideal as it provides no signals for tone, and our brains backfill this information in a biased manner.

You emailed me less than an hour ago (I found out about it here, just now) and then tried to dunk on me for not replying. I think we can save ourselves some time and disengage.

Respond to my skywriting on the subject of error-handling, you coward! That plane was expensive and my opinions are important!

If I'd seen the skywriting before you complained, I would have! I like getting email--- err, skywriting messages! But I only check the, uh, sky a couple times a day!

That's reasonable. My bad.

I think part of the reason for the response is that people tend to just use libraries and assume they will work without reading the documentation or the code and when that strategy backfires they are surprised.

At another level: this is also caused by the fact that most users of said libraries would not be able to write those libraries in the first place and so are not qualified to read/understand the the code.

I mean, the behavior we're talking about here is in fact documented; they don't have to read the code. Every mainstream language in the world (that supports socket programming) has a setting to enable or disable Nagle, so it's not like it's hard to know where to look.

Likely the first time when they realize something is up is when it doesn't work to their expectations. I can see why though: the Go eco-system, and many others besides treats including dependencies as a black box operation, and with auto completion you can include a library and start using it without ever really understanding it, its design trade-offs, default settings and so on. They might show up briefly by name during some dependencies installation process but all it takes is one level of indirection to hide the presence of some library fairly effectively.

Just like someone who installs a refrigerator likely has no idea how a heatpump works, they just need a box that is cold and as long as it is cold they're happy. Cue them surprised when the box starts working in unpredictable ways when the environment temperature changes outside of the design parameters.

Back up a step: why would anyone who's never read documentation assume something like Nagle's algorithm is in effect? I call send(), I expect data to be sent.

Indeed. But the devil is in the details and many networking protocols have layer upon layer of fixes to ensure that things normally speaking go smoothly. Depart from the beaten path and you are most likely going to find some of your assumptions challenged.

One of the more frequent occurrences is the silent fragmentation and re-assembly of packets and/or the attempts to transmit packets that exceed the MTU. These are all but guaranteed to lead to surprising outcomes and much headscratching.

A name like send_but_make_sure_you_read_the_documentation() would have probably been more appropriate but it's a bit unwieldy, and in the default case it is precisely the silent activation of various algorithms to fix common problems that allows you to get away with calling it 'send()' in the first place.

Probably because it’s the default in most other scenarios you’d call send (other languages etc).

So having a rare inverted default is bad for intuition.

I took the implication here to be that the kinds of people who don't read documentation don't know what Nagle is or have any expectations about it to begin with.

Because it's a tradeoff. The author touches on this in the last sentence:

> Here’s the thing though, would you rather your user wait 200ms, or 40s to download a few megabytes on an otherwise gigabit connection?

Though I'd phrase it as "would you rather add 200ms of latency to every request, or take 40s to download a few megabytes when you're on an extremely unreliable wifi network and the application isn't doing any buffering?"

In the use cases that Go was designed for, it probably makes sense to set the default to do poorly in the latter case in order to get the latency win. And if that's not the case for a given application, it can set the option to the other value.

Delayed ACK seems like the better default to me, whether it is telnet or web servers, network programming is almost always request response. Delaying the ACK so that part of that response is ready seems like the correct choice. In today's network programming how often is tinygram really an issue?

In this case I would consider the bug to be git lfs. Even if Nagle's was enabled I would still consider it a bug, because of the needless syscall overhead of doing 50 byte writes.

Reply on news.ycombinator.com