Maybe call it "removes as much friction as possible compared to local functions" but please don't call it "feels [exactly] like local functions". It's simply impossible and pretending so will just cause trouble.
Over the course of my short 20-year career, I've encountered far too many applications that accumulated terrible performance or reliability problems that were directly attributable to their authors feeling a bit too comfortable about getting chatty with the slow, unreliable remote procedure calls.
By contrast, the most successful microservices shop I ever worked at was very conscientious about doing nothing to hide the messiness of RPC. (Ease the implementation, yes, but not hide the messiness.) And that seemed to foster a culture of being much more thoughtful about interaction patterns and data flows and how they might contribute to or detract from the overall performance and stability of the system.
However, I've found the abstraction to be useful when someone _has_ already made the decision to refactor their service out to be a separate service.
Sure, `sprintf` will run synchronously and finish in known time, but `read` may as well end up waiting for seconds for an HDD to spin up, or for a mounted NFS filesystem to respond. `read` is not any different from an RPC call. And, in fact, `read` can fail in just the same ways as an RPC call can. So can `write`, which is why `sync` exists.
Software ignoring these potential issues, and pretending they don't exist, is the source of many unnecessary software hangs and crashes, such as Windows Explorer crashing if your SMB share responds too slowly or Linux being unable to unmount an NFS share if the connection breaks badly.
The solution IMO is that we should finally stop hiding asynchronicity behind synchronous facades. If coroutines, promises, async programming, etc are first-class features, it becomes much easier to cleanly model these potential errors and in turn handle them correctly.
And once you have that, an rpc call
async fn rpcMakePurchase(user: &User, order: &Order) -> Result<Purchase>
async fn read(path: impl AsRef<Path>) -> Result<Vec<u8>>There is still a small difference though: when I execute a write and sync, then I'm pretty sure it either succeeded or didn't. I'm not sure if there is any way for the disk to say "don't know" which the OS will then pass to my program. Even if, I guess such a case is so rare that it is probably excluded in any error-model. Like cosmic rays. And I can always retry, since it's very very unlikely that there is a connection issue only between my program and the disk but not between other programs and that disk - since they run on the same machine.
Over a network things are different, because the connection might be gone for a long time and know I'm not aware of the state and I can't check it. I can also not tell if the connection might come back, the machine I was talking to might have burned down. If I, myself, burn down, then I don't need to worry anyways.
That's the difference. This is specific for write/sync though - as you explained it, just because things are running on the same machine does not mean things are necessarily more reliable than a "remote" call.
When modeling rpc calls as async calls returning a result, you can be just as sure that it has completed (or failed) as with a local call.
And considering write can use an underlying mounted network share, a spun-down HDD, or a thumb drive on a failing USB port, the same failure modes exist in either case.
In either case you're just submitting commands to another device's queue (and waiting for a result) over a hot-pluggable connection.
This is not true for networks (the internet).
Your code still has to handle the same failure modes regardless.
I've had many situations where e.g. reading locally cached files on a smartphone eMMC had worse latency and throughput than doing the same read over the network on the server. The cache actually made performance worse in every way.
If it were the same doing this over a network, then we could have a distributed, consistent postgres. What would I give for that. :-)
Regarding having a distributed postgres, that's not an issue actually. As long as all writing workers have access to the same synchronization primitives you can easily use postgres with e.g. k8s volumes in single writer, multiple reader mode. And single master, multiple read replica postgres deployments are common too.
The guarantees you're talking about aren't given by the storage implementation, but by the fact that all write workers run on the same machine.
Do you see what I meant?
So why do you think that is?
http://armstrongonsoftware.blogspot.com/2008/05/road-we-didn...
Bye, Joe.
We didn't know we were going to be posted on HN, and we're still iterating on copy that "lands". I don't think we've found the words yet.
I agree with the premise that RPC has markedly different characteristics. The actual implementation [1] makes it clear that you're connecting through a client interface.
Thanks for the feedback on calling it "removes as much friction as possible compared to local functions". I think we'll trend this way.
[1] https://docs.differential.dev/getting-started/quick-start/#3...
Nonetheless, what you built sounds quite interesting: can you say anything more about the language, tech stack, etc…?
The result was good but I still don't like transparent rpc (on large dev teams anyway).
In particular, if the "network" is something local to one machine, pcie or some cross-chip fabric thing, where the failure modes of the "network" are the box falls over, then RPC works as one might hope. You don't have to mangle functions to take account of extra network errors because there are none.
The specific use case I'm interested in is heterogenous compute on shared memory machines. Code running on a GPU can call `int rc = fprintf(stderr, "oh dear");` and have no idea whether that runs code on a CPU or not. Code running on a CPU can call some linear algebra function and either run it locally or on a GPU, unobservable to the caller.
That is a good trick. It does make for an interesting debugging experience where you lose track of where code is executing.
----
There's a less esoteric application in the area of the OP for pure functions where you don't need the result immediately. That goes something like:
1. Call the function and get back a promise or similar. All runs locally, no failures
2. Something in the background maybe does the work for you
3. Ask for the result. If it's not available yet, do the work locally
That has the same zero failure modes property because it'll ultimately run all the work locally if the network offloading never works out. It only really makes sense if the functions can be called multiple times (e.g. locally and remote at the same time) without doing harm, i.e. pure functions. It costs latency and gains throughput. For some batch computation work that is great.
So you'd probably need the "not available, but already 'stolen' case". For example, the current machine could look for work to steal itself. But this probably will increase latency a lot, so it would only make sense if the batches are huge.
The problem with the "already scheduled" pattern is when the machine holding the task falls over.
Not a good choice for bank transactions. A good option for something like a distributed build farm - it doesn't matter hugely if you compile some piece of program repeatedly, but it matters a lot if you send the only copy of some source to a machine that immediately goes offline.
Mainly bringing it up as a case in which RPC works beautifully - remote machine failures can be completely hidden by a willingness to do the work locally instead.
But exactly once processing is viable.
With Differential, the implementation does allow for idempotency in cases where you need exactly-once processing. And it does handle the "not available, but already 'stolen' case".
Disclaimer: I'm an author.
Fair enough. Maybe we shouldn't call it remote procedure calls then, because it's not remote anymore. The word remote exists to make the distinction compared to local. If there is no distinction in a given context, there is no need to use that word from the beginning.
There is another big restriction. The sender and receiver have to be in the same process. Else it's possible that the receiver does not exist and there is no way to start the receiver back up.
I generally agree with the fact pretending it's like a local function will cause trouble down the track. After all, you're introducing a number of extra failure modes.
The client library however does give some protections to the developer through the type system - although it's not perfect.
Hence the wording "feel like". It's not actually local function calls. But it abstracts away a lot of the complexity.