Hacker News Reader

Comment by CodesInChaos | Hacker News Reader

I find it surprising how few protocols (besides Cap'n Proto) have promise pipelining. The only other example I can think of is 9p, but that's not a general purpose protocol.

https://capnproto.org/news/2013-12-13-promise-pipelining-cap...

View on HN

As neat as it is I guess it's hard optimize the backend for it compared to explicitly grouping the queries. I imagine a bespoke RPC call that results in a single SQL query is better than several pipelined but separate RPC calls, for example.

But even still, you would think it would be more popular.

If you're thinking strictly about stateless backends that just convert every request into a SQL query, then yeah, promise pipelining might not be very helpful.

I think where it shines is when interacting with stateful services. I think part of the reason everyone tries to make everything stateless is because we don't have good protocols for managing state. Cap'n Proto RPC is actually quite good at it.

The issue is that having per-session/transaction state on the server makes load balancing requests more difficult; especially when that state is long-lived.

While it's true that load-balancing long-lived resources is harder than short-lived, a lot of the difficulty of load balancing with stateful servers is actually in the protocol, because you somehow have to make sure subsequent requests for the same state land on the correct server.

Cap'n Proto actually does really well with this basic difficulty, because it treats object references as a first-class thing. When you create some state, you receive back a reference to the state, and you can make subsequent requests on that reference. The load balancer can see that this has happened, even if it doesn't know the details of the application, because object references are marked as such at the RPC layer independent of schema. Whereas in a system that returns some sort of "object ID" as a string, and expects you to pass that ID back to the server on subsequent requests, the load balancer is not going to have any idea what's going on, unless you do extra work to teach the load balancer about your protocol.

Wow that is super interesting. Thanks for sharing.

If it's the same backend handling multiple chained requests that happen to use the same database, it could in turn build a single big SQL query to generate the result(s). I used to write stuff like this all the time (not specifically Cap'n Proto servers, but general functional expression -> SQL -> here's your answer engines).

> I find it surprising how few protocols (besides Cap'n Proto) have promise pipelining.

Pipelining is a bad idea. It reifies object instances, and thus makes robust implementation much harder. You no longer make stateless calls, but you are running functions with particular object instances.

And you immediately start getting problems. Basically, Client Joe calls Service A and then pass the promised result of the call to Service B. So that Service B will have to do a remote call to Service A to retrieve the result of the promise.

This creates immediate complications with security boundaries (what is your delegation model?). But what's even worse, it removes the backpressure. Client Joe can make thousands of calls to Service A, and then pass the not-yet-materialized results to Service B. Which will then time out because Service A is being DDoS-ed.

I didn't know 9p had promise pipelining!

Or more specifically, it seems to have client-chosen file descriptors, so the client can open a file, then immediately send a read on that file, and if the open fails, the read will also fail (with EBADF). Awesome!

This is great, but "promise pipelining" also needs support in the client. Are there 9p clients which support promise pipelining? For example, if the user issues several walks, they're all sent before waiting for the reply to the first walk?

Also, it only has promise pipelining for file descriptors. That gives you a lot, definitely, but if for example you wanted to read every file in a directory, you'd want to be able to issue a read and then walk to the result of that read. Which 9p doesn't seem to support. (I actually support this in my own remote syscall protocol library thing, rsyscall :) )

There is also CapnP’s moral ancestor CapTP[1]/VatTP aka Pluribus developed to accompany Mark Miller’s E language (yes, it’s a pun, there is also a gadget called an “unum” in there). For deeper genealogy—including a reference to Barbara Liskov for promise pipelining and a number of other relevant ideas in the CLU extension Argus—see his thesis[2].

(If I’m not misremembering, Mark Miller later wrote the promise proposal for JavaScript, except the planned extension for RPC never materialized and instead we got async/await, which don’t seem compatible with pipelining.)

The more recent attempts to make a distributed capability system in the image of E, like Spritely Goblins[3] and the OCapN effort[4], also try for pipelining, so maybe if you hang out on cap-talk[5] you’ll hear about a couple of other protocols that do it, if not ones with any real-world usage.

(And I again reiterate that, neat as it is, promise pipelining seems to require programming with actual explicit promises, and at this point it’s well-established how gnarly that can get.)

One idea that I find interesting and little-known from the other side—event loops and cooperatively concurrent “active objects”—is “causality IDs”[6] from DCOM/COM+ as a means of controlling reentrancy, see CoGetCurrentLogicalThreadId[7] in the Microsoft documentation and the discussion of CALLTYPE_TOPLEVEL_CALLPENDING in Effective COM[8]—I think they later tried to sell this as a new feature in Win8/UWP’s ASTAs[9]?

[1] http://erights.org/elib/distrib/captp/index.html

[2] http://erights.org/talks/thesis/index.html

[3] https://spritely.institute/goblins/

[4] https://github.com/ocapn/ocapn

[5] https://groups.google.com/g/captalk/

[6] https://learn.microsoft.com/openspecs/windows_protocols/ms-d...

[7] https://learn.microsoft.com/windows/win32/api/combaseapi/nf-...

[8] https://archive.org/details/effectivecom50wa00boxd/page/150

[9] https://devblogs.microsoft.com/oldnewthing/20210224-00/?p=10...

Without knowing how exactly capnproto promise pipelining works, when I thought about it, I was concerned about cases like reading a directory and stating everything in it, or getting back two response values and wanting to pass only one to the next call. The latter could be made to work, I guess, but the former depends on eg the number of values in the result list.

In the actual implementation, when making a pipelined call on a result X, you actually say something like "Call X.foo.bar.baz()" -- that is, you can specify a nested property of the results which names the object that you actually want to operate on.

At present, the only operation allowed here is reading a nested property, and that seems to solve 99% of use cases. But one could imagine allowing other operations, like "take the Nth element of this array" or even "apply this call to all elements in the array, returning an array of results".

Not being able to do N+1 queries would be a feature in my book.

io_uring supports that too, although not a network protocol.

Last time I checked (a couple of years ago) they wanted to use eBPF to handle this sort of problem. Did they end up doing something simpler?

Yes. io_uring lets you issue multiple syscalls together, with the result from some being parameters for others.

Redis transactions [1] also apply pipelining, but AFAICT there is no practical way to use them for implementing generic RPC.

[1] https://redis.com/ebook/part-2-core-concepts/chapter-4-keepi...

Does the pipelining in Redis allow you to have the second command depend on the result of the first command?

No, but for that use case you have EVAL which execute an arbitrary lua script on the server.

https://redis.io/commands/eval/

That assumes you know & can generate the complete pipeline ahead of time. The elegance of promise pipelining is that your pipeline can also be asynchronously grown.

Reply on news.ycombinator.com