Hacker News Reader

Comment by paxys | Hacker News Reader

Unless the alternative is a 100% backwards compatible drop-in replacement for TCP, even starting to discuss it is pointless. And if it is compatible at that level, it will just bring with it all the pitfalls that we want to replace in the first place.

The TCP/IP stack works because I don't need to care about what environment the two processes that need to communicate are running in. They could both be on my local machine, or in my home network, or communicating over the internet, or some random intranet, in a data center, across continents, on any OS or any kind of device...it simply does not matter. "Just do a ground-up rewrite of your entire software stack and you'll get a guaranteed 5% efficiency gain" isn't the bullet proof argument that people who come up with these alternatives seem to think.

View on HN

I don't agree. A subset of high performance applications in a datacenter can use a new protocol while still supporting TCP for other applications. I'm not saying this is easy, or even worth it, but it isn't all or nothing.

I think the parent commenter would be pretty surprised how much code would really need a rewrite. Google and Amazon do not have hundreds of thousands of engineers messing around in the mud with sockets and connections and IP addresses and DNS. There's a service mesh and a standardised RPC framework. You say "get me the X service" and you make RPCs to it. Whether they're transported over HTTP, UDP, unix domain sockets, or local function calls is fully abstracted.

If you need to go that fast why not implement a layer 2 protocol?

The point of these abstractions is that they are insurance. We pay taxes on best case scenarios all the time in order to avoid or clamp worst case scenarios. When industries start chasing that last 5% by gambling on removing resiliency, that usually ends poorly for the rest of us. See also train lines in the US.

> If you need to go that fast why not implement a layer 2 protocol?

How? Datacenters aren't a single link, you need to route packets.

> The point of these abstractions is that they are insurance. We pay taxes on best case scenarios all the time in order to avoid or clamp worst case scenarios. When industries start chasing that last 5% by gambling on removing resiliency, that usually ends poorly for the rest of us.

I'm not sure what this means. There are other transport level protocols already, UDP is in fairly regular use. Is your argument that TCP offers us some insurance that Homa will not?

> If you need to go that fast why not implement a layer 2 protocol?

Because datacenter is not single LAN segment ? You need to route it

Do they need to replace a lot of code?

I'd suspect that the tcp bits are hidden in the rpc layer anyway, be it grpc or whatever

I think the point is: Goals of "A subset of high performance applications" is not going to _replace_ TCP unless its works OK for everyone else.

Additional M.O. protocol cool, but replace TCP with it?

Is quic 100% backwards compatible drop-in replacement for tcp? No. Is it pointless to discuss. Also no.

And DC applications are far more easy to switch over than general internet, less middleboxes to screw you over, generally better performing networks, and more tightly controlled hosts.

Honestly, it wouldn't be all that farfetched to have AWS implement QUIC over SRD for squeezing the last perf drops out.

IPPROTO_SCTP does this today. Just have to convince middleware/box vendors to support internet protocol.

Where this conversation might have some points would be that you don't typically have these middle boxes within a single data center, so you could try to ignore them.

Until you remember that you have datacenters that need to talk to each other and then this strategy doesn't work.

You'll also need to convince several operating system maintainers to build/optimise their implementation first.

I wouldn't have any trouble ignoring middlebox software (or adding a TCP fallback with a big warning that something suspicious is interfering with the connection) but Windows and macOS still lack proper SCTP support, Linux' SCTP support has some performance issues and usermode raw sockets will probably need to bypass several OS sandboxes to be viable.

That said, in server to server connections SCTP can probably be used just fine.

…and Microsoft and Apple if your endpoints include Windows, IOS or MacOS. All of those have 3rd party drivers but I don’t think any of them come with SCTP support out of the box.

It's a no-go if they did anyway at the moment, the number of endpoints that's behind a NAT box that only knows about UDP and TCP is huge.

With the rise of IPv6 that may soon no longer be a problem. Plus, server to server communications usually aren't behind NAT boxes.

IPv6 is such a pain in the ass to deploy that people developed IPv6 NAT. For more than a decade the protocols to delegate an IPv6 subnet to a customer at the edge were missing. With PPPoE the situation was basically that you'd get a dynamic /64 on the PPPoE link (what a waste) and then have to manually configure the local IPv6 subnet on the CPE. IPv6's failure was largely self inflicted by failing to make it easy for ISPs to implement.

DHCPv6 and its prefix delegation extensions have been out since 2003. It took longer for IPv4 to receive DHCP support than it took IPv6 to get DHCPv6 (which was complicated by the stateless vs stateful debate). Back when IPv6 was standardized, DHCP was just two years old and wasn't a common method for assigning IP addresses in ISP networks.

IPv6 definitely got off on a rocky start but none of that has mattered for decades now. If IPv6 deployment is still such a pain, that's because of either your choices of the choices of your preferred vendors.

Except that wireless router manufacturers use ancient SDKs that are based on Linux kernels and tools that were typically 5+ years out of date as they were frozen by the hardware designers early in silicon development. It took a long time for upstream tool updates to trickle all the way down the hardware development path and into shipping consumer gear. IPv6 support for an ISP was still really difficult in 2010 because of that despite the protocols being released in 2003. I know because I tested a bunch of consumer gear back then and working IPv6 support was still hit and miss. Things are far better today.

This is only really true of external connections, for orgs with large degrees of control over their internal stack this can make sense.

I wonder if we need to replace TCP in the data centers or if what data centers really need is RPC (which ia covered well with QUIC)

Right, I've been saying this for a while, it seems that many of the hyperscalers/etc made the mistake of trying to utilize TCP (and maybe HTTP/etc) for their (generic) RPC mechanisms. Which is hardly the best plan, when what is actually needed is just a reliable datagram protocol without a bunch of connection state.

So, while the actual RPC protocol has its own issues, the one thing they got right was the ability for the portmapper to indicate UDP vs TCP as the transport on a service basis. There have been a few improved generic RPC mechanisms, and I don't really understand why some of these places feel the need to "replace TCP" when really what they need is a more formalized RPC mechanism that can set/detect the datagram reliability and pick varying levels of protocol retry/etc as needed.

Many applications don't use TCP/IP directly, but RPC (or as others have pointed out, protobufs). Then someone needs just to adapt the RPC layer to support the new network protocol.

Most applications don't care about TCP any more than they care about IP.

Correct. I agree. Well said.

Suggested alternative title: "Is it time to replace oxygen in the atmosphere?"

Reply on news.ycombinator.com