Rescuing REST from the API Winter | Hacker News Reader

Rescuing REST from the API Winter

intercoolerjs.org

55 pointscarsongross55 comments

I don't understand why HATEOAS is incompatible with JSON. There's not a link/form standard like HTML, but there are still the standard HTTP verbs, and if you include a URL for each resource does that not meet the requirement?

Stepping back a bit, I feel like people put Fielding's dissertation on a pedestal like it's some sort of perfect theoretical foundation that must be adopted in its entirety or not at all, but I see it more as simply exploring the idea of how to leverage HTTP instead of treating it as a dumb transport with no purpose but to add a layer of indirection like SOAP did. It was definitely a prescient thesis, but it also wasn't yet battle-tested in practice.

I think HATEOAS didn't make the cut as a universally adopted practice because it doesn't provide all that much benefit. Sure it's nice in theory to be able to navigate around an API without any external queues, but even if you have in-built hypermedia standards like HTML forms, the purpose of an API is to perform some function, therefore you will always need some external semantic description of what you are doing. Contrast this with something like SQL where there is a mathematical foundation for manipulating rows and columns that is completely independent of any semantic weight the data may carry.

I am definitely keeping an open mind that I might be missing some key insight, but to me there's just not a huge untapped promise of REST, it's just that we learned what works and what doesn't, and frankly it's worlds better than the bad old days of SOAP/WSDL.

JSON certainly isn't incompatible with HATEOAS in theory, but we have enough observational evidence at this point to conclude that it will not be used in practice, on any meaningful scale.

From the article:

HATEOAS is a constraint of the REST application architecture that distinguishes it from most other network application architectures.

The is the under-appreciated aspect of regular old HTML is that it satisfies this requirement, and that makes HTML-based applications fundamentally different than (nearly all) JSON-based applications.

I certainly don't under-appreciate HTML. I was there at the University of Minnesota when the WWW appeared in our Gopher and set the world on fire, and I've made a career out of it ever since.

But the magic of HTML is in the browser where you click around. Having a standard for form equivalents and links in a JSON API doesn't pack the same punch because there is little purpose to have freeform consumption of an API the same way that we have freeform consumption of web pages by humans. In theory I understand that tools could be built around rigid adherence to RESTful principles, and that this could allow some sort of usage without out-of-band documentation, but REST principles per se are not a specification for something that you can build tooling against. For that purpose I feel efforts like JSON-API (http://jsonapi.org) are doing an admirable job, and whether it is technically RESTful seems like a minor footnote compared to whether it's making good decisions that A) prevent bikeshedding and B) enable better tooling.

> But the magic of HTML is in the browser where you click around. Having a standard for form equivalents and links in a JSON API doesn't pack the same punch because there is little purpose to have freeform consumption of an API the same way that we have freeform consumption of web pages by humans.

I think that's the whole point of the original article: If we're creating an interface for human consumption (a UI), then REST is eminently relevant, and we shouldn't forget the power of HTML-over-HTTP as its most widely-supported realization. But if we're creating an interface for programmatic consumption (an API), then REST is more problematic and we shouldn't feel guilty for not achieving its ideals.

You just have to keep in mind the goal of REST: self navigation. Given a known document, it should be possible to automatically navigate the graph of related documents. REST, technically, is a poor choice for an API.

REST assumes, at least in my opinion, a human driver on the consumption side. A rendered page is for a human. REST coalesces HTTP. HTTP came first. To get anything useful from an API requires as much context as deriving meaning from an HTML document. REST, by itself, does not try to give a semantic meaning to the data. An API does.

So take the useful and leave the useless. If you want to extend JSON to link, fine. Just make sure you document your new type. If you want to use just the noun + verb paradigm, go ahead. Please be consistent. Please document your assumptions.

The article assumes having HATOAS is somehow more important than having an API. Maybe it's not.

SQL and JSON APIs share the same vision (dis)regarding HATOAS. A foreign ID in SQL looks like a JSON field with an ID in it. HATEOAS links are more like storing SQL statements as a foreign key.

So if you think SQL did it right, so did JSON without HATEOAS.

If HATEOAS was such a nice feature, everybody would use it in JSON APIs. Its proponents always say the world would be better with it, but the world disagrees.

I'd say that HATEOAS links are like foreign key constraints. In a JSON document, you get an ID, but there's no semantic information about what it refers to, it's just a number. In SQL, on the other hand, the table has that metadata, and so it can tell you that the number refers to a specific record of table <X> - just like the link in HATEOAS encodes the specific document it's referring to.

Some databases don't even support explicit foreign key constraints, and SQL doesn't make their declaration mandatory. And that metadata is not in the resultset, you get an id instead.

So SQL dbs will happily let you join with the wrong table. Also foreign key constraints don't play well with sharded SQL dbs.

For these reasons, I don't see SQL as a good parallel to HATEOAS.

Maybe an ORM would be closer to what HATEOAS brings... It's all about impedance mismatch I think... but HATEOAS is no more a definite solution than ORMs.

Fielding's work isn't specific to HTTP.

It was just agreeable with it.

This is an ad for intercooler.js.

If I make an API request to a remote server, I want data back, which I will then process in some way. It might not be displayed to a user at all. I don't want HTML to display to the user. That's too hard to process in a program. Also, if it returns HTML, it will probably have an ad in it, or Javascript, or a tracker, or something else obnoxious.

If you're accessing an API which deals with financial data, you usually plan to graph it or compare it or issue a trade order. The server has no idea what you want the data for, and doesn't have a need to know. You don't want HTML back; then you'd have to scrape the numbers out of that.

JSON in, JSON out as an RPC mechanism is useful. It's easier to use than SOAP, and few SOAP servers seem to actually obey their published schema anyway. (Run a checking SOAP client and call something complicated to see this.)

Well, it's kind of an ad of course (although intercooler is free and open source, with no company offering any services around it) but there is a deeper point here.

I see a lot of confusion in these comments: I'm not advocating HTML as a general API for use with native mobile clients, SPAs, etc. And I'm saying, explicitly, that REST-ful JSON, as implemented by most developers, has proven a bad choice for that need. And I think that is because HATEOAS has never found a natural expression in JSON.

So I'm saying: if you want to follow REST-ful principles for your web application, you should abandon a data-level API entirely, and move to a partial HTML model instead, where HTML allows you to naturally leverage HATEOAS without even really thinking about it.

Ad, deep insight, whatever. It's a thing.

The thing is, if one wants to follow REST-ful principles in web applications, one can just write JSON documents with links. The fix is not hard nor complicated - which, to me, shows that the problem is not that people really want to use REST but can't due to JSON, and must be something else.

In my opinion, people don't really want to use REST, either because they don't really understand it (as seen by the number of RPC APIs being described as "RESTful") or they just don't think it's the best solution for their problem.

That said, Intercooler.js can still be a very useful library, but I think it might be successful despite REST, not because of it.

I don't agree: the fix is hard, because not only do you have to pick one of the many at-war non-standard standards for your links in JSON, but then clients must figure out how to interpret the links. I think the empirical evidence of the last two decades overwhelmingly supports this conclusion.

This is what HTML gives developers: they don't even realize they are satisfying the HATEOAS constraint because it is simply so natural.

I think its just a naming issue/confusion. JSON APIs are not RESTful ? ok, fine. Lets rename it to JSON API architecture and move on. Nobody is going to flip and start sending HTMLs now in responses just because its not "RESTful". Who care? It works. Lets fix it when it breaks. End of story!

So it turns out an admin interface on top of a CRUD API is a good design. Weirdly, that's what the pragmatists have been doing for 10+ years.

I don't know why we have to do so much navel-gazing on this topic. REST is just a loose set of guidelines for invoking operations on, and exchanging data with a remote server in a more or less stateless fashion. If json works, use it. If xml or yaml work use one of them. Provide both if you want, and if it turns out that sending back an html snippet is the right thing to do then don't feel like you have to burn your HN t-shirts or something.

Server returning HTML responses? Yuck! How do you deal with that on mobile platforms such as iOS and Android? No, thank you! Plain old JSON is just fine.

I know: a server returning html.

It will never work.

EDIT: I've fallen into the ad internetum trap; I shouldn't be so flippant. Sorry about that.

What I am saying is that, for JSON APIs that are needed (for, say, mobile clients) that REST is probably not a great architecture, and it will and should move towards something more like SQL: a single RPC URL with a very rich query and update syntax.

Where REST can still deliver a tremendous amount of value is in pure HTML applications, and intercooler helps make that possible within the context of a modern web application.

Indeed. Things like GraphQL and even OData should hopefully take over the vague idea of "JSON available at a resource" which is needlessly complicated for most APIs.

I have great hope for OData, but I just wish the team would focus more on getting it to work with ASP.NET 5 - it's in RC1 already.... As it stands I've decided to not use it at work, simply because I have no idea of when or if it will be a first class citizen of the system.

I never liked the ('orthodox') RESTful approach when building web apps and I didn't understand why so many codebases I've worked with seemed to bend over backwards to be as RESTful as possible with (seemingly) no benefit. I feel like I now have a slightly better understanding of why I felt this way (and some more 'authoritative' support, I guess).

Not that i love REST, but for an app that needs to work offline, boring ass REST is a pretty straight forward way to sync the mobile and remote databases.

To be honest, your occasional comment promoting Intercooler.js became rather annoying to me, to the extent that I even thought about doing another 'HN Comments parody' just so I could feature you prominently (among others).

But this comment somehow is the first time I have a better perspective on what it is exactly you're pushing, and whether it's my fault for dismissing you earlier or not, I just wanted to let you know that while I don't have a use for your tool right now, I feel like I get it and I even agree with your reasoning.

Furthermore, I applaud anyone who build something and seems passionate about it, and I hate falling into the trap of being negative. My apologies thinking that way.

(that said, if I were to do a comment parody, you'd be featured. just with a more friendly thought behind it.)

(lol) Yeah, I recognize how quixotic and annoying I can be about the whole thing: I certainly deserve parody. I'm a lone developer up in Sacramento trying to compete for mindshare with Facebook and Google. It's ridiculous and extremely funny.

Thanks for being fair-minded about it. :)

Like GraphQL?

I was under the impression that iOS and Android both support HTML right out of the box. Do they no longer bundle Web browsers, or are you suggesting some other drawback that is critical to your particular application?

I assume they are using the same JSON API for their client side web app and their native mobile apps. Is there a compelling argument to move template rendering to the server if the native applications don't render using a HTML DOM?

Not for mobile apps. An app can display html via a web view but that has already proven to be a terrible experience.

There's a reason JSON APIs have become more popular. Besides, clear separation of data from view means the data can be used by other third-party services.

The future of mobile apps will be native client-side view rendering from SPAs like React Native, Angular2 Universal. Not serving HTML views.

It all depends on your context. If you are making a website, HTML on the mobile. If you're making an API, you can use JSON, just don't call it REST.

People seem to overlook RPC over HTTP. There is nothing wrong with that model. Look at the early RPC implementations. They look like objects with a set of wlverbs. A URI can be treated logically as an object. /users is an object. Implicit ops are the HTTP verbs. How you move between objects is probably API specific. If you are using JSON, it is API specific because JSON currently lacks linking capacity.

I haven't used intercooler.js myself, but I think it's a really beautiful example of REST's conception of "Code-On-Demand": It's not just downloading an RPC-style client into the browser, it's enhancing the browser with a richer hypertext implementation. Since hypertext is the core of REST, intercooler.js is embracing and enhancing REST itself in the process.

Pardon my ignorance, but what is wrong with a JSON response and how does this library solve that problem?

I would suggest reading the article and the linked articles in it, and then thinking about it for a bit.

I do not mean this flippantly.

I wrote that question after reading the article, I am just not really understanding why an HTML response would be a better alternative to JSON. I thought the whole reason JSON was used for RESTful API's due to the client-server principal.

After rereading the article, if I understand what you are saying, the reason why the library was written was to address two failings with JSON based API's: They do not follow the HATEOAS requirement of REST and a JSON API may not have the same flexibility as raw SQL.

To be honest, I am still a little confused and not really understanding what problem is being solved by the library... I am sorry.

OK, let me try to summarize:

REST does not work well for JSON APIs

This is because HATEOAS for JSON APIs never worked out

JSON APIs will continue to move away from REST and towards RPC and SQL-like architectures

However, if you wish to adopt a REST-ful architecture for your web application (and I think it is a good one for many applications), there is a good way to do so: have your server return bits of HTML, which naturally allows for HATEOS, and use a library like intercooler.js to fully embrace the concept.

Does that clarify?

I did little research and now I understand why I was confused. JSON lacks hypermedia support which makes it violate HATEAOS (http://restcookbook.com/Mediatypes/json/ ); inherently, there can never be a RESTful, JSON API beca use it violates this principle. Everything I know about REST is wrong! Thanks for taking time to write out your response and this article, I was missing this one piece and couldn't wrap my head around it.

I'm not sure this is correct. Even the link provided does not exclude JSON as a media type. Just because there aren't (m)any examples of RESTful JSON API's doesn't mean there can't be.

My understanding is that as long as you standardize the way in which hypermedia is communicated through the API, it could be RESTful.

The REST paper desires to reuse existing doc types. If you create some linkable JSON, you've made a new type. So now everyone that wants to interact with your data has to first use your type. They will have to say Accepts:x-new-linky-json. Your type will probably be a bit different than someone else's linking JSON. So now we've got type explosion.

I don't advocate the creation of a new media type.

My take on REST is that for an API to be RESTful, the API definition should really be the definition of a hypermedia format. The format definition then informs the interpretation of a hypermedia response such that state transitions can be made by the client. If our encoding of a hypermedia response is a JSON document, the format definition should describe how we can interpret that response to perform some action.

If the service we provide is as general as a Web Page, then using a standardised format like HTML makes a huge amount of sense. If the service we provide is an API to some specific functionality, it seems reasonable to me that we can define the interpretation of some specific media type as a part of the specific hypermedia format that drives the interface.

Then again my interpretation of REST could be completely backwards and I would appreciate corrections.

You cannot interpret a media type. They have a definite meaning.JSON does not have a linking ability. To make it, you've created a new mime type.

REST is an interesting architecture. It can inform your decision. If you don't follow its prescriptions, you are not following REST. You are merely informed by it. That is fine. Don't confuse the two.

I don't think you're ignorant. To me, it is niave for anyone to think that HTML meets all needs for response data. Having to support everything that HTML can do relay limits the practicality. Similarly, HTML can't support every kind of response you might want to provide, and then you're forced into parsing or transforming the HTML. It's stupid.

You are missing the point of the article. I am not suggesting that people use HTML APIs in place of JSON APIs. I am suggesting that they should use something else for that:

It’s time to admit it: the REST-haters are right. REST does not make for a great raw data-level API architecture, and efforts like GraphQL and RPC-like architectures are likely to end up producing something more usable for complicated data needs.

Rather I am suggesting that the REST-ful architecture can be rescued from API use, using HATEOS and annotated HTML in web applications.

This article took a long time to write and, while I don't expect people to agree with me on the value of REST/HATEOS or the viability of intercooler as a web application library, I do think it is worth taking time to understand my argument.

The time you spent on this is much appreciated. Everyone is trying to get a handle on the best way to do web development.

I also appreciate people that either (1) buck the SPA/JS trend for the purpose of trying to make the web more accessible, or (2) develop APIs that are at least usable in theory by clients that don't limit accessibility.

However, when you rely on a combination of JS and HTML that isn't accessible and APIs that can't be used easily by non-browser clients, I don't care whether you are trying to adhere to Roy Fielding's vision or not, I can't go for that.

It reminds me a little of Tim Berners Lee pushing for https://en.wikipedia.org/wiki/Semantic_Web with dreams of https://en.wikipedia.org/wiki/SPARQL supporting databases and https://en.wikipedia.org/wiki/RDF_query_language defining everything. Really smart guy. Really neat idea of how things could have worked. But the real world is not like that, and it isn't filling the needs of today's web developers in a smart, fun, and practical way.

If we're going to "fix" web development, let's just start over.

How about defining data to view/feel/hear and APIs to use in a way that doesn't enforce the method that you use to interact with them? What if webpages were JSON instead, and that JSON could embed definitions of available services? What if all of the web was decentralized so we weren't reliant on anything except the device you used to host this JSON and its services and it was easily scalable across peers that decided to host it?

That's the sort of future I'd be in favor of.

I have returned RDF/N3 from my web services for years. And transformed that data into a Javascript graph of objects on the client with the library available at https://github.com/RubenVerborgh/N3.js.

It is very flexible. Each object has a unique identifier. That identifier is dereferencable as a URL. Each object has a type (because the Semantic Web is all about typing your data, and applying your code on data of a given type). The graph can be traversed in any way I want on the client.

Honestly, this makes things quite simple.

I like the idea behind the semantic web and if everyone had the bandwidth to commit resources to it, it would definitely add additional abilities to the web. But, in my experience, the time spent on adding RDF support wasn't worth the adoption or limited gains in site usage.

What are its usage metrics in your site(s)? Does it appear that there is significant growth in its adoption?

Facebook's has offered RDF for a while. I've not heard of its use outside of academic circles.

RDF has no adoption. JSON is the winner. Let's try to understand why: Because JSON.parse() is instantly available on the client. On the contrary, RDF.parse() (or more precisely N3.parse()) is available on the client only if you use that obscure library I mentionned above. Noone knows the library, so noone uses the N3.parse(). Ok. End of discussion.

Now my own feeling: RDF (and especially its N3 dialect) is the only description language I know of that serializes and deserializes graphs with no specific code, and has a good JS lib for client-side consumption.

JSON (or XML, or any other tree descrption language) requires extra effort whenever you use it to serialize/deserialize/traverse a graph structure. That is why i think they suck. And that's why I prefer RDF/N3.

WTF are native mobile clients supposed to do with an endpoint that returns HTML?

This is nothing but an ad for a product I'm not even going to mention.

REST is not the end-of-all designs for an API. It's a specific type of API that has tremendous benefits as long as you need CRUD and simple actions. And if Json does not float your boat, then return whatever you like, REST does not dictate the format of the data. But one thing i certain, I'll continue to not use HTML as a format for an API. It's just the wrong spec for exchanging data. If Json is not enough, XML most certainly is. It's just more work, but it's sound and simple once the schema is defined.

The reports of REST's death are greatly exaggerated.

This article is pretty close to what I think on the matter.

W3C isn't a monolithic organisation, there are many working groups that aren't really connected with each other. And some of those groups had a declarative vision for the Web, it's about time we re-examined those ideas.

J. Cole said it,

    "Don't save her
    She don't wanna be saved
    Don't save her
    She don't wanna be saved".

REST is a convention. Like all conventions it excels in some cases and confuses in others. If it adds ambiguity to your API, you might consider not using REST. But no matter what don't take the path of most resistance and embrace a convention just because..

I really wanted to like this concept, but I think I have identified my philosophical nit:

Client side templating takes advantage of the structure of data to predefine reusable components which minimize bandwidth and latency.

Intercooler could provide a class system that predefines inheritable element configurations, but I feel like verbosity is their core tenet.

After gzipping and so on, I would contend that most partial HTML payloads (outside the original frame of a web application) compare relatively favorably with the equivalent JSON payloads.

Microsoft's WCF Data Services is HATEOAS, and works beautifully from that point of view. Unfortunately it is a little slow and verbose because your payload becomes bloated, and has fallen out of favor with both MS and developers much like most dev frameworks from MS.

So intercooler.js = RJS / SJR?

https://signalvnoise.com/posts/3697-server-generated-javascr...

Sigh. Seems like the author has never really come across JSON-LD.

Reply on news.ycombinator.com