Let's stop building APIs around a network hack (2017)

Most advice I've heard from REST aficionados tends to be terrible for simplicity, correctness and performance. All in favor of an abstract set of goals, most of which never materialize. This is no exception.

Articles like this talk about a world I don’t understand anyway. It’s like the rest/hateois etc. stuff. I just make an MVC action that does what I need in a way that makes sense and call it from the browser. If shit is slow, I profile it. No highfalutin design philosophy!

Both in terms of latency (to be fair, probably pretty small in comparison), and in terms of knowing that all the data you fetched is consistent (because you fetched it in the same transaction).

Your reply assumes that API requests need to be made in sequence. If you need 3 resources, and you make 3 requests, you are not waiting for 1 then 2 then 3. 1 and 2 and 3 are all coming back along the multiplexed connection and none are blocking the other. That is generally what a lot of compound documents are doing, but sure the article also talks about when you need to know some information from 1 before you can call 2 or 3.

- Server Push - since axed from some browsers, but the idea there is to proactively push the resource to you before you request it, because you know people often request Bar after requesting Foo. That means its not 1s it could be 100ms or less. The GraphQL community took this idea and created @defer.

- If people ALWAYS need to request 1 and 2 and 3, that API sounds like it was overly normalized and not too well planned. Maybe its more of a "Data Store" than an API.

It also talks about how by making endpoints smaller and more targeted, the cachability of one resource does not impact another, so you dont need to wait 1s for each requests, you just skip a lot of the requests. This article covers a lot of the same points.

https://www.fastly.com/blog/optimise-api-cache-improved-perf...

In something truly RESTful (with HATEOAS and media types and all that jazz), the links between resources are only discoverable dynamically, and the client traverses that resource graph interactively. As the article explains, that works a lot better when you aren't scared of the per-request network overhead. But it still fundamentally limits the amount of context provided to the server on each interaction, preventing the server from optimizing for each request beyond guessing and pre-fetching what it thinks a client will need. (And the pre-fetched data has to live somewhere, meaning you've just created a cache and all its concomitant issues.)

Taking GraphQL as an alternative, the resource graph is statically known -- you can effectively fetch the schema just once. You can then traverse through many resource relationships and fetch many disparate units of data in a single request, and the backend can see that structure and decide how to efficiently gather all of that data.

Even besides the interaction context, doing REST properly (instead of a faux-REST actually-RPC HTTP API) can be pretty challenging. How many REST APIs are founded on media types instead of statically-determined URLs and methods? [0] If you're just doing HTTP RPC, you more-than-likely have a statically defined API schema, but no way to leverage that static knowledge for efficiency. (Instead you get clients coupling to the specific API endpoints instead of to the media types, making it not just static but also rigid.)

Also, HTTP/2 Push (mentioned in the article) seems to be in a weird place right now. [1]

[0] https://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypert...

[1] https://evertpot.com/http-2-push-is-dead/

you speak like there's anything that exists outside of a 70km-long band in California

    GET /invoices?invoicesQuery=criteria
    GET /invoices/orders?invoicesQuery=criteria

I would expect server push to be basically unused until something comes around to automate it. It seems totally plausible to me, for example, that NGINX might add some logic to automatically keep track of which resources are requested after which other resources, and, for example, notice that "/style.css" is requested after "/" 99.9% of the time, and therefore automatically push the stylesheet.

It says a lot about Google if their approach to this stuff is to build something which requires the entire world of web servers to adapt, and then rip out the feature if it doesn't take off right away. If they were serious about it, getting everyone to restructure how their infrastructure works should've been treated as a decades-long project.

This just reinforces my existing view of not trusting Google for anything important. They just can't keep at a project. Imagine being the sucker who suggested that a business should spend a load of time restructuring their back-end infrastructure to support server push, just for Google to rip it out.

The main problem is that tertiary data slows down primary data.

Put simply, when you request 3 things, it might take a little while for the server to find process the response, create all the JSON, shove it all over the wire, and be picked up by your client application.

The user has been sat there waiting for all three things to come back at once.

Real world example: a social network full of posts and images is failing to load because the application wanted to know if there was any space left in a meeting room nearby.

If you have 3 different things they probably have different cacheability, so let them declare their own cacheability and let the client only bother fetching them if they need to. Show the things you have when you get them. Progressively enhance the page. Don't just show nothing.

There's a lot of solutions to various problems with compound document systems like JSON:API and I've written about those too: https://apisyouwonthate.com/blog/making-the-most-of-jsonapi

But the core purpose of compound documents was the same thinking as CSS combination and Image Sprites, most of which have gone away now, and I think compound documents should too. They were a handy hack that had downsides and its no longer necessary to mess with them.

All errors are 200 unless they are HTTP errors. The backend error is under the errors key in the json.

All post requests.

REST as an architecture has a lot more in common with how we users traverse the web than how we imagine our machines to query an API. We go from page to page and interpret the content on each page to decide which link to follow next. The HTML media type is understood by our browsers, so we're presented with a consistent way of traversing, submitting forms, etc. that behaves consistently across any site that provides an HTML resource. And a website can update its pages as much as it wants, and as long as it's still providing HTML, a user can learn to navigate it again.

Of course, that genericity and reliance on human learning is exactly what makes HTML alone not a good media type for machine-to-machine REST APIs, but that's why media types are (supposed to be) a cornerstone of REST: they describe the schematic structure of a general kind of resource, specifying how such resources may relate to other resources and what other kinds of data they might contain, but leave it up to the individual resources to actually make use of those facilities (and up to the client to reason against those facilities).

But try finding a "RESTful" API that actually does that. Most of the ones I've seen actually use a bank of statically-known URLs that report a content-type of JSON whose actual fields are statically-known by separate specification. It doesn't really matter whether you're using HTTP or not, that's just RPC.

I'm very interested in getting GraphQL features into http and wasn't aware of these prior proposals. A quick web search didn't turn up anything for me on them either. Thank you!

This reminds me of the time when people took a format meant for sprinkling some metadata here and there on a piece of a large document, and started using it primarily for writing "documents" where markup/data ratio is > 2.

I'm of course talking about XML here.

Development teams go "ok, we need to design a REST API", and, due to the inherent architectural characteristics of REST (large-grained) clashing with their own needs (small-grained) what they end up with is something more like some Frankenstein-esque REST-RPC monster.

Beyond whether its RESTful API or not (grey area) it adds headaches, and is rather unnecessary with good API design anyway, which is what this article was about.

More importantly, compound documents mess with the "cacheability" of resources. A resource cannot declare its own cacheability (which REST suggests) through the uniform interface when its munged into a mismatch of other data. A foo and a bar and a baz are now all cached in together under the same headers with the same /foo?include=bar,baz identifer, meaning that if a bar is likely to change often you have to request the whole thing again. This is one of many reasons compound documents are a pain in the ass, but there are plenty more.

Fastly wrote about all this: https://www.fastly.com/blog/optimise-api-cache-improved-perf...

> The current praise of HTTP/2 as a major win for REST APIs shows how differently "REST" is understood by its proponents today compared to its original meaning. If doing REST today is about doing the opposite of what it was intended to do, it also raises the question why do we believe we get the same benefits REST did for its original definition? A quick read shows we don't, but the cultural ingraining in favor of "believing" is huge.

This whole paragraph makes no sense. I have been making APIs since ~2008 so let's not act like I'm brand new at this.

The point I was making here is that in the past people have moaned about REST because "making lots of requests is slow", which is a common comment from people who generally build their APIs poorly not taking into consideration what the consumers would be doing and just flopped out a very normalized database and forced clients to construct their own data models from that. It's also a concern from people who havent got the first clue what HATEOAS means or how it can help abstract state from multiple clients up into the backend. They just see it as "having to make more requests".

Now that the fear of multiple requests is subsiding, the exact same concepts that made REST beneficial for many APIs in the HTTP/1.x past exist now, they're just quicker in HTTP/2. And they're quicker in HTTP/3. REST and HTTP just get quicker and more useful over time.

Anyway, you sound like you might enjoy this article: https://apisyouwonthate.com/blog/rest-and-hypermedia-in-2019

Also when 5 becomes 8, 9, 10.... entire APIs come grinding to a halt because people are trying to download the entire database in one single request instead of just making a few calls.

Basically, trimming a few bits off of a request is not overall quicker than helping clients not need to make as many requests by designing an API well enough that resources can live on their own and declare their own cachability.

Designing your API resources so that primary data can be fetched and displayed before you start mucking around with secondary and tertiary data is also important. When you talk about being slower for the user, is it actually slower for a real human doing real things on the site. For example, I've seen GraphQL queries loading a social media feed, and list of friends for a little box on the side which didnt matter, and a list of comments on posts that havent even been expanded yet so now user could see it, and querying to see if there is any availability in conference rooms even if theyve not shown any interest in booking a conference room. The clients wanted to avoid making multiple calls because ooooh thats bad, but the whole page was just a blank white screen until all that pointless data came back. Querying all of those resources seperately over the same HTTP/2 connection and letting the UI progressively enhance would absolutely be quicker than mushing it all into a single giant HTTP request but that's what everyone does. All the time. It's insane.

Again, Fastly wrote about all of this very well. This article is one in a series which doesn't stand as well on its own.

https://www.fastly.com/blog/optimise-api-cache-improved-perf...