RPC vs. TCP (Redux) | Hacker News Reader

RPC vs. TCP (Redux)

systemsapproach.substack.com

67 pointssigna1113 comments

I have a hard time wrapping my mind around this whole line of discussion because it seems to treat as a binary something that is a continuum. It is not the case that we have "TCP" and "RPC" as some sort of cleanly separable discussion, or at least, if they do mean it as some very cleanly definable term it doesn't seem to have ever gotten a discussion.

RPC, to my mind, isn't really a thing of its own. It's a particular popular pattern for using messages. You can use that pattern poorly or well on TCP. You may need to use another underlying protocol for any of a variety of reasons to get another performance profile out of it. You may be in a situation where you need all the guarantees of TCP anyhow, in which case the room for improvement is somewhat marginal (although non-zero, especially if you want to be on the cell network).

I'm not particularly convinced there's a need or even "room" for a pure RPC protocol, on the grounds that it'd probably end up much like SOAP in that it can only serve a particular set of needs and the benefit of a custom protocol for those needs could well be outweighed by the costs of the stampede of developers jamming everything into the new protocol regardless of their needs, most of which would mismatch. The field of "RPC" is already very diverse and to my eyes, people generally have satisfactory solutions available to them.

It's not like it's even possible to build something that would both replace generalized RPC on the internet and service supercomputer mesh networks.

If TCP was the only option, yes, many needs would be going unmet, but it already isn't. If I do an analysis and determine that TCP is my core problem, it seems to me I already have an abundance of choices serving many points on the continuum. I don't see what trying to lift up one of those points as some sort of standard would particularly accomplish that isn't already being accomplished in the real world. And it isn't even RPC qua RPC driving that per se; you can have all sorts of niche needs that drive you off of TCP, of which RPC doesn't even strike me as a particularly distinguished case.

RPC when you get down to it just isn't that complicated or really even that interesting; you send a message, you expect a 1-to-1 message back in reply. Of all the message patterns, it's just about the simplest other than unilateral broadcast without any reply at all.

RPC over SCTP could be interesting, not sure of any libraries that support it out of the box. Could also be hell for enterprises/firewall admins.

You can obviously build an RPC protocol on top of TCP, people have been doing it for decades. However there are things that are difficult to efficiently implement on top of TCP (multiplexing different logical streams over the same connection, message priorities) in the TCP model, and some things that are impossible to implement over TCP (for example, you don't really have any ability to influence what happens during congestion control scenarios). There are also a bunch of things about TCP that are fundamentally inefficient on modern hardware.

Do these things matter if you're running some RPC service over the internet with a Python backend? Almost certainly not. But if you read John Ousterhout's paper or watch his Netdev talk^[1] it's also pretty clear that this isn't the use case he's been talking about. He's talking about RPC in the datacenter, where in theory modern hardware is capable of delivering RPC messages with microsecond scale latency but in practice no RPC systems built on TCP come anywhere close to that. The Netdev talk I've linked below is really good and John makes a really thorough presentation of his ideas and what he's actually trying to optimize for.

[1]: https://www.youtube.com/watch?v=o2HBHckrdQc

Not the GP, but I think the confusion stems from comparing TCP to RPC as if they are at odds with each other. TCP (Transmission Control Protocol) is a delivery protocol (i.e. how can we have a conversation between two places). RPC (Remote Procedure Call) describes how the messages should be treated (i.e. how each side should talk).

So, RPC vs TCP doesn't make sense, even just looking at their names; a new protocol designed for RPC (maybe an RPCP) vs RPC over TCP does make sense.

This is just a name collision. Rpc in this context (that is transport layer protocols) just means a request /response pattern.

A better category name would be nice.

(Compare middleware - it's a concept that is pretty broad and has a lot of meanings over time but has been coopted to a subset that tends to mean "parse http headers and do things before the main endpoint handler")

It's a misnomer, really. HTTP is request/response. If you want a non-TCP request/response protocol, QUIC is quite good.

> A better category name would be nice.

Sure, but if QUIC is good enough, then we don't need a new category name yet.

In earlier posts on this topic (HOMA) we saw that TFA wants a new upper layer protocol to replace TCP and which is strictly a request/response protocol. They call that an RPC. RPC is a misnomer that invites all sorts of quibbling, so they would do better to not call it an RPC.

New ULPs have trouble getting deployed due to bad middleboxen. It can work in a data center, but you have to deal with provisioning carefully when you opt to toss the congestion control belt and suspenders. More importantly, QUIC exists and is probably good enough, and without evidence that it's not I find HOMA and similar proposals to be possibly premature optimization.

You should check out Lustre RPC. It's basically RPC over RDMA, and based on Portals. It uses RDMA for bulk data. The actual RPC call/reply headers are all essentially a flavor of FlatBuffers before that existed, using C and recipient-makes-right for byte endianness.

If you want an RPC that is not over TCP and which takes advantage of RDMA, then Lustre RPC is what you want to study first. I wouldn't necessarily copy Lustre RPC but only because one might want to use a better encoding for headers and a better language than C. But I would copy Lustre RPC in every other way unless a different choice of RDMA technology made that impossible.

Interesting that gRPC is too slow for the latencies they're aiming at, maybe Cap'n Proto would be a better match.

He doesn't go into details on how they handle packet loss, but the paper[1] says:

  Receivers use a simple timeout-based mechanism
  to detect lost packets. If a long time period (a few milliseconds)
  elapses without additional packets arriving for a message, the
  receiver sends a RESEND packet that identifies the first range
  of missing bytes; the sender will then retransmit those bytes.

Their implementation uses 15 ms as a default, so even crossing regions e.g. between us-west-1 and us-west-2 would time out [2]. So they're really only targetting communication within a single datacenter, this is not a protocol for "the Internet". Maybe annoying if you need to switch protocols depending on where your target server is located at.

[1] https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18....

[2] https://www.cloudping.co/grid

Previous article on HN: https://news.ycombinator.com/item?id=34871710

Thanks! Macroexpanded:

It's TCP vs. RPC All over Again - https://news.ycombinator.com/item?id=34871710 - Feb 2023 (112 comments)

Excellent, now I don't have to make similar comments.

> No one would argue that implementing a reliable byte stream service, bundled with a fair congestion control algorithm, should be left to the application.

User-land TCP stacks exist. Sure, that's not "leaving it to the application", but still.

Re: HPC and RDMA, yes, if you want every ounce of performance (HPC) then you want RDMA. Lustre was doing that, what, 20+ years ago? But the price you pay is that you get no congestion control, and you have to plan everything. That works great in certain data centers -- it works great if you're CERN recording petabytes of data from one particle physics experiment. It also works great in data centers in general if you don't exceed limits.

> Saving the best for last, my third observation is the consequence of an exchange between David Reed and John Ousterhout. David argued that “RPC was one big reason for creating UDP as a datagram ‘process addressing’ support layer” and that “...argument passing and waiting for a return was way too high level for a transport protocol.”

Er, UDP goes back at least to 1980, and as such it predates the RPCs of the 80s (Apollo Domain, Sun's ONC RPC, etc).

Reply on news.ycombinator.com