Telefunc: Remote Functions instead of API

* semantics of what the request / response looks like and how failures communicate

* how are the APIs composed? Not the local function call, but the functionality calling the API implies (see cap’n’proto’s page on composable remote APIs - it’s really brilliant).

* can you communicate across programming languages?

* how is ownership of remote resources managed?

* how is routing / dispatch of API calls managed?

For example “fetch” in JS land looks a hell of a lot to me like a normal function call to me. And I certainly then wrapped it with easier to read APIs that looked more normal and handled routing and error handling for me. The main difference with HTTP is that RPC systems traditionally autogenerate a bunch of code for you. But in HTTP land you now have efforts like OpenAPI to do similar things. So HTTP isn’t RPC but the autogenerated code from a description of the API makes it RPC? That gets you into the ship of Theseus paradox which isn’t helpful. At what amount of wrapping it with local functional calls does distributed API calls transform into RPC?

To me it turns out that async / sync is a key distinction and the original sin of RPC in the 80s. It’s also why I view the coloring problem as a good thing (at least given where the tech stack is today). Making functions that may be doing I/O stand out and needing thought is a good thing. The problem with many RPC/IPC mechanisms from the 80s is they tried to make the code seem synchronous not that it looks like a function call. But I haven’t seen any RPC systems where async vs sync is the distinguishing characteristic (eg cap’n’proto and fetch are both async).

To me, HTTP, REST, SOAP, COM, gRPC, cap’n’proto, JSON-RPC, XPC, Mach etc all try to solve the same problem of distributed communication and processing, just by making different trade offs.

HTTP tried to standardize RPC semantics but you can clearly see it’s optimized particularly around static document retrieval and typical browser tasks. I’ll buy that static document retrieval need not be classified as a remote API but there’s a lot of non static document retrieval that’s layered on the same semantics that is proper API calls (notably REST) tries to get everything to conform to that so that middleware has some convention and you get browser interop. It works surprisingly well because a good majority of apps are CRUD and don’t need anything more.

JSON-RPC tries to solve routing/dispatch to be more automated if I recall correctly because REST is such a cluster (look at all the routing libraries which to me are an anti pattern). + JSON-RPC is particularly easy in JS which is usually the language on one side of the transaction.

COM solves all of the above and things like distributed ownership of resources, except it historically didn’t have a good security model (I don’t know it well enough to know what’s improved there but I don’t assume it’s stayed static since the 90s even if it hasn’t found traction outside Microsoft).

They all try to solve different parts of the problem or have different sets of tradeoffs, but to me it’s fundamentally the same problem. It’s the same reason I don’t view distributed systems as something that requires different machines, CPUs, processes or anything about geography. You can have distributed systems within one process, you can have it within processes on the same CPU, you can have it between CPUs on the same device, etc. Heck, to me even the kernel and user space form a distributed system with syscalls as a primitive RPC mechanism. JNI is a form of RPC too although you’ve seen traditional systems like gRPC supplant it even within process because it moves complexity out of the JNI layer into a simpler system with easier to follow rules.

Now you can sometimes makes simplifying assumptions to reduce complexity, reduce power usage etc. and not all distributed systems necessarily have the same sets of problems. But fundamentally to me it’s all the same problem space and why I don’t distinguish the terminology so much.

I'll agree with you that it's not a trivial change, far from it. You need switching your mindset from local to remote, from procedures to agent messages, from exceptions to non-local error-handling, but it remains quite simple and readable.

I feel that Telefunc is bringing developers a good chunk of the way towards that goal. And, who know, maybe there is another simple model that is not exactly agents, and that would also work nicely.

The RPC debate is confused because the two sides of the debate have different ideas of what RPC is. The anti-RPC crowd is arguing about RPC systems from the 80's, which tried to make RPC look exactly like a local function call, including blocking until the call was complete. The pro-RPC crowd is mostly speaking from experience using modern RPC systems which do not do this; they use asynchronous callback APIs, have some mechanism for reporting network errors, etc. Modern RPC systems are called RPC because they encourage you to organize your network API conceptually into something like function calls, but they certainly do try to hide the fact that there is a network here that can fail in all the ways networks fail.

On windows it was DCom, then COM+ then remoting the WCF then who knows I lost track.

Rest APIs are simple and easily debuggable, magic remote api layers are not.

That’s why REST is still prevalent even in websockets had better better performance parameters (and in my testing it did have performance advantages) yet 7 years after my testing hire many sites are running websockets?

See the backend for front end pattern.

Debuggabilty wins.

    We’re calling this the cloud, but the first iteration of “the cloud” started with storage enabling efficient, reliable access of data storage being decoupled from individual nodes

Just in general I do see in my history this which came through HN a while ago. I never read any of that but it seems a lot like what we did in computer science in uni from a cursory glance: http://infolab.stanford.edu/~ullman/focs.html

You are right in that technology changes and different things that were not possible become possible or become mainstream accessible. What I don't like is the complete ignorance of what came before because of marketing. Don't get me wrong, I get it in that sense. It only sells well when you make everyone believe it's the best thing since sliced bread and that you just invented it out of thin air. I don't have to like that though.

We start out with sequential reads being faster for example. Think tape drives and how file formats and search algorithms got optimized for sequential reads being faster. Then hard drives come along where you can suddenly do much faster 'random reads' simply because searching is so much faster in comparison and people take advantage of it. But then you build enough functionality on top of that sudden ability to do random reads relatively fast that it does become the bottleneck and you instead organize things on your HDD such that they result in sequential reads instead. Along come SSDs, which are so much faster overall that you don't have to any longer. Until it repeats and your workloads bottleneck on even an SSDs abilities. But the idea of organizing your files in a certain way or taking advantage of the read characteristics of your storage aren't completely novel at all.

Or take databases. We start out with accessing data by key only, because it's faster. You need to know your access pattern beforehand if you want things to be fast. Someone comes along and invents relational databases and it becomes possible to actually do that in a 'fast enough' way as well because of hardware advances. Suddenly someone else comes along way later and re-invents accessing data by key only if you need 'real speed' (aka NoSQL databases).

Or like in another thread where someone claimed 'the cloud' invented node independent re-attachable networked storage. No they didn't. I'm not sure if that's the earliest but it's the earliest I know of, which are EMC^2 Symmetrix and that was way pre-cloud i.e. in the 1990s. SANs are/were a thing for a long time. Mainstream access to such technologies by simply renting an EC2 instance w/ EBS volumes is something the cloud did give us though.

Primarily, async exposes that the call you are dealing with goes over the network. You can no longer accidentally put a blocking rpc call in the UI thread.

You can easily send multiple concurrent requests without fuzz.

It has built in primitives to add timeouts on any call, even chained ones, without having to rely on complicated thread-watchdogs or checkpointing.

I'm more than open to support Zod & co, but I'd like to see a concrete use case for it.

Maybe you want something more barebones, maybe you want to leverage existing protocols. These days, it's kinda the wild west out there in terms of web standards it seems, so go nuts I guess.