Nobody Understands REST or HTTP | Hacker News Reader

Nobody Understands REST or HTTP

blog.steveklabnik.com

100 pointsshinvee70 comments

This was posted 7 months ago on HN (with 510 points!): http://news.ycombinator.com/item?id=2724488

Very lengthy discussion at that time.

I believe the author of this has refined/updated some of this thoughts since that first post.

I believe you are right. Also he is writting his own book: http://getsomere.st/ (linked from the part 2)

The part about "accept-language" doesn't address the real problem. If I copy the url to a piece of content that is available in 2 languages and send it to someone else, I want them to read it in the same language as I originally read it. Sadly, I cannot copy Accept-Headers in a fashion that "normal" people understand. To me, resources written in 2 different languages may be instances of the same thing, but not the same resource.

At worst, the page in question uses automatic translation (not so uncommon when it comes to large knowledge bases), so the translation might be odd.

Basically, according to the article, Wikipedia has it all wrong.

I think different approaches are suitable for different situations. Quite often I think different translations of the same thing should be treated as different (but related) resources.

Sure, but the article makes it look like language encoded in the url is all wrong, while I think that (most of the time) it is exactly what you want. ACCEPT_LANGUAGE is a perfect for educated guessing of a users preferred language, but not for referencing an article.

There are numerous opinions on the subject of RESTful apis. The author of this post makes many good points, but I, like many of you disagree with his opinion on versioning.

While putting the version into the header is clever, it reduces the usability of your api. When you are designing a RESTful api you typically want to design your api so that it is simple and easy to implement. Nothing is easier than being able to explore an api via a web browser. If you make the user specify versions in the header than they will have to install a browser plugin to explore your api.

You have to consider that you aren't always "selling" to other developers who understand http headers. You could be "selling" to non technical project managers and CEOs who simply don't understand http headers. They do understand URIs, though. So if you can provide these people with a URI that just works, and they can poke around and see data they have a greater chance of understanding and getting excited about your API.

This says versioning in the URI is wrong. Why? I much prefer separating my API versions into different files at the dispatcher level. Doing it with the header would be a mess in most frameworks, and I don't see what it hurts.

Also, what about custom HTTP verbs? Many times I need something more than GET/PUT/POST/DELETE. What happens then? I haven't seen anyone talk about that.

What I've come to realise is that if you try to make an API of any sort, somebody, somewhere, will write a post about how you're defiling the HTTP protocol.

Totally true. For example, this author would have problem's with reddit's API, where you indicate that you are requesting JSON by adding .json to any URL. And you know what? The world doesn't come crashing down.

There's nothing wrong with doing that. A resource can have multiple URLs and multiple representations. That's fine, in the right circumstances.

You say that, but a lot of people scream bloody murder about it. And this exemplifies one of the major problems for people like me that are trying to learn how to create REST APIs and understand why to bother doing so in the first place. The problem is that even people who seem to know what they are talking about fundamentally disagree on many things. And Fielding's dissertation does not cover many implementation details. It's like interpreting the constitution trying to figure out how it should all happen in the real world.

Some of the points of contention:

(1) Should api version go in the URL or accept header?

(2) Should representation type go in the url or accept header?

(3) Are custom HTTP verbs okay?

(4) Should GET, POST, PUT and DELETE map one-to-one to CRUD? Most say no, but other seeming authorities say it's fine.

(5) Should you ever return a URL template, and a set of entity identifiers that can be plugged in, or should you only return the fully formed URLs?

(6) What is the difference between a URL and a URI? (And all you who say this one is dead simple and I'm an idiot for not knowing, see if your answers actually agree with each other)

(7) Are two different language translations of a document different resources, or are they different representations of the same resource?

(8) If two different language translations are just different representations of the same resource, how should the client indicate which version it wants?

And of course some people say that a few of these questions aren't even within the scope of REST anyway.

Some answers, may be wrong but this is how it makes sense to me:

1. Possibly both (the one in the URL being the version of the URL-space and the one in the Header being the more fine grained Representation Version).

2. It depends. For simplicity it often makes sense to put it in the URL. For more generality headers can be better.

3. No

4. They don't have to. They usually do though and it makes most sense when they do.

5. Fully formed URLs are simpler, so go with those if you can. But sometimes you'll need more.

6. URLs are a subset of URIs. All URLs are URIs. URLs give the location of the resource whereas URIs just have to identify it (but could also give the location of course.

7. It depends. I lean towards different but I'm no authority.

8. Using the Accept-Language header. But I'm guessing this is not always the best way to do things.

With a RESTful API you don't tend to need IDs most of the time, you just use URLs. So having all the versioning info in the URL is not so great, you change the version and suddenly all the URLs aren't valid anymore.

But more important that is upgrading a version on an API may not be an all or nothing thing. You might want to start using the new features of the API on one resource type but you aren't ready to upgrade your usage on everything else. If the version is in the API you'll have to take apart and put back together urls to get the right ones, this is logic you don't want to have to encode into your client. If on the other hand you use Accept headers to do versioning you can have as fine grained control as you need.

Regarding the custom HTTP verbs it may seem like you need those at first but in practice you really don't and there is almost always a good clean way of doing things that doesn't break anything (or so I've found). I find that the solution is usually to introduce another resource or two, the transactions example from the article is a perfect example of this.

I know it sometimes seems like this is all abstract stuff that has no impact on the real world but it does make sense eventually! It's really lovely to use a properly RESTful API :)

You have swayed me, old man. My next API will use headers for versioning, thank you. As far as the custom verbs are concerned, you might be right, I'll need to think about it when I next need to design an API. Thanks again!

Huh? The whole point of having a version in the URL is so you can roll out a new set of URLs and their corresponding endpoints without touching the old ones. When v2 of the API comes out, you still support v1. That's no different than using headers. If you sunset or change an API call, the header isn't going to save you any trouble. In fact, since the client now won't get a 404, but rather something else, the cost of debugging goes up pretty substantially.

But how do you deal with the partial upgrade scenario? I might want to use a few of the new features in my client code but not have the time to upgrade (and test!) everything just yet.

Of course version numbers in URLs can have their place too. But I think they should only really be used in situations where you have upgraded so much that the original URL space just makes no sense. And if you can avoid huge breaking upgrades like that through good initial design then all the better :)

The same way you'd handle it on a per-header basis . . . you just use the new API version in the URL where needed. Sorry, I feel like I must be missing something here.

You are a little. In a properly RESTful API you simply don't construct URLs most of the time. Instead you follow links. If you change the URL structure you break the links. If that's what you want then a version number in the URL makes sense, but in a properly RESTful API that's probably not what you want.

I think for a lot of people the reason they don't see the benefit of things from REST is they try to use them in isolation. You say "this is useless for my RPC style API!" and you are right.

But, if you follow a link and that link is generated by the server, I still don't see how encoding a version number in the URL makes things any harder.

FWIW, I've made an effort to follow the purity of it, but the APIs I see that try to do true ReST are insanely obtuse. If you know of any real world case studies where a provider went from "fake" ReST to true ReST, I'd love to read it. The contrived examples are not helping me out in the comprehension department.

How frequently do you think restful apis move to a new version?

Typically you don't release a new version until you need to do significant changes to the entire layout of your api. In which case the old version is not even relevant. With this in mind, you would not want to promote using a new version of the api for one resource with an old version of the api for another resource.

Just thinking about it sounds dirty. Your migration strategy should include sufficient time for you to assist your clients to move to your newer versions.

I think little upgrades happen all the time. And sometimes you need a way to handle those.

You also have to consider which of these 2 are easier to understand:

api.example.com/v2/account

or api.example.com/account Accept: application/vnd.steveklabnik-v2+json

And I'm not talking about how easy it is for Developers to understand. I'm talking about how easy it is for Everybody to understand.

I'd argue the former is a lot easier/cleaner to document as well.

you are somewhat missing the point. the idea of proper REST is that you NEVER should generate your urls. you take them from the resources. now suppose you have an application that consumes RESTful API.

if you do it "wrong" you probably store record ids in your local db and then access stuff from the API by constructing your own urls.

a more proper way of doing it is to store the compete resource urls and not just ids. but in this case you have a version update problem. When you release a new version of the app that supports new version of the API you will now need to go over all the stored resource urls and update the version, probably doing some regexing etc.

with version in the headers you can store full resource urls and not have to change them every time api version changes.

If I want to share a link to your API, and you're using accept headers for versioning, how do share that URL?

I think that not only is using accept headers worse, I think it is flat out wrong. That's not what the accept header is for. It's for client specific things only.

The version of the API you need is NOT client specific.

I agree, lots of the Header only people live in a Utopian view of services. Many times these services are needed at the javascript, even plugin (flash, unity, etc) level that don't work well with hidden headers since all they have is a url and a body to make it work. The Academic view of REST is ideal but usage in the real world is difficulty with headers only.

A situation that may arise is people not even aware they are using the wrong version of the api. And which one do you default to? Do you start with the version header always or do you add it into the system on version 2? Then do you require the version header? What happens if you didn't add that in version 1? Which one do you default to the new one? Is it really easier to change one line header than one line service prefix?

It is much much easier when maintaining a large service with more uri focused api routes/actions/verbs at times. It still is defined as rest but has some rpc elements to it. You don't even have to go down to the the model like /api/profile/[uuid]. You can do things like /api/signup /api/login and match those sensible abstractions into the rest api that lives above the model layer. This is always more flexible for minimizing versions and allowing core model changes without having to change much and provides a better consumer experience of the api. This is more of a REST and RPC mix that works well if you have to work with things beyond just backend services such as games, interactives, scripted experiences etc.

I make services fully RESTful when I can but I have to build services that run in scripted clients without easy access to headers, this is where header based versioning is more difficult for the consumer of the service. What I have been doing is looking first for a header, then for the url version, the routes are abstractions anyways so I route those accordingly.

I am liberal in what I accept, conservative in what I send like a good service should be.

Can you elaborate on Unity not working well with headers? I've just started doing some web-stuff with it and found that so far it's been okay.

Services are a bit painful in Unity due to the WWW class: http://unity3d.com/support/documentation/ScriptReference/WWW...

You can use System.Net.* but if you want support across iOS, Android, Web, and Desktop you need to use WWW. Which does not allow you to read or set headers.

I have decent services working with it in JSON using LitJson but it is something they should work on and has prevented me from using headers much in Unity. Even the WWWForm class is weak but there are hidden methods to set and read headers, just they don't work on all platforms yet.

My biggest hiccup in making true REST has actually been in games where web standards and tools are still pretty fresh. So many C/C++/C#/Python/Lua etc proprietary web libs that to make things work you have to have some fallback to uri based states or resources. I typically implement that as a fallback i.e. the version 1 api with headers or url route to the same place.

Ahh yeah, I noticed that things are different from platform to platform. Thankfully we mostly work with the web player rather than the mobile platforms, so it's been pretty painless so far. I'm really loving using coroutines with the web stuff, they fit nicely together.

Thanks for the info :)

It depends what you want to share though. Are you sharing the location of a resource or are you sharing a specific version of that resource?

If I'm viewing some resource in some client that supports version X and I send it to you and you view it in a client that supports version Y (maybe this happens automatically, maybe new versions are in the process of being rolled out, maybe your on the mobile client that hasn't been updated yet) isn't it better that my client gets a representation of it that it can understand?

(I don't think there's one answer for everything, I think both can make sense in different situations. But I think versioning via Headers can be very powerful and can interact well with of RESTful types of things you might want to do, it would certainly make no sense in an RPC style API)

You're using the wrong frameworks :)

Custom verbs aren't really necessary. You can do some of your "method" naming in the URI. One example could be: /sum?number=1&number=2 which, when a GET is issued, will return a representation of 3.

If you need to perform some action that can be expected to take time, you can issue a POST request, which can give you a new resource that reports back on its current state. An example could be: POST /printer with the request body containing the document to print. It could assign a URI that tells you the status of the document (location in queue) and you can DELETE it if you no longer want to print the document.

It sounds like you are making an API design choice based on how easy it is for you to implement something internally. What about your client's ease?

Version in the URL requires them to update the URLs everywhere. Version in the accept header is likely a one line change somewhere.

This is not necessarily true. A client could easily have the api version segment of the url in a central configuration location.

The problem is designing all this nonsense into the bottom layer in the first place. I've never heard a good argument as to why any of this is better than just starting with raw sockets for an app talking to a server.

Take HATEOAS for example; a completely academic waste of time. Writing a tiny bit of API documentation isn't hard. Caching is an awful thing to be designing in without knowing the specific application. Verbs are a hack bolted on to make hideous form/link driven applications work when the user reloads or moves back/forward.

Don't downvote, comment. Or admit to yourself that you can't respond to my criticism.

Wow, what a weak crowd. No substantive criticism, just downvotes. The HN crowd should please refrain from arguing on the Internet about design until they've learned some basic systems programming.

Please downvote this comment too. The more you downvote me without commenting, the more you confirm my superior skills in system design.

Except that when you follow all of REST, including HATEOAS, that part of the API documentation is FREE. And the caching is FREE, because its already been designed for you. Which is the whole point.

What? How can you say REST is going to give you a bunch of stuff "for free" when you haven't even seen the application. How do you know you're even going to get what you want? This is the central conceit here.

Again, downvotes but no comments. How embarrassing for HN that the crowd here can't justify their position.

How can you say (TCP|Unix|C) is going to give you a bunch of stuff "for free" when you haven't even seen the application. How do you know you're even going to get what you want?

You don't, of course, but it's usually a pretty reasonable assumption that the effort involved in implementing your own (reliable stream sockets|scheduler and memory manager|high level language compiler) will eclipse severalfold any advantage from doing so. If you don't find that the same obtains from using HTTP and hypermedia then that's your position and you're welcome to it, but don't go assuming your experience is universal.

>You don't, of course, but it's usually a pretty reasonable assumption that the effort involved in implementing your own (reliable stream sockets|scheduler and memory manager|high level language compiler) will eclipse severalfold any advantage from doing so.

Wrong. I implement network protocols for a living, and I use often use UDP because TCP is not sufficient for the purpose. So yeah, this is the problem: your conceit that your upfront design is going to be "enough" for everything.

See where I said "usually"? And where I said your experience is valid but don't assume it's universal? Right

I never said my experience was universal, nor did I imply it. So what's your point?

If "I've never heard a good argument", "a completely academic waste of time" and "why should anyone non-academic building actual systems care what it means" aren't implications that your experience is valid and others' experiences aren't, then I concede and am happy to admit that you have totally Won The Argument if it makes you feel better

In either case, I'm done here.

The fact that you think it us "usually" a valid assumption is exactly the conceit I am talking about. It's only "usually" assumed to be true because the people who have different requirements don't come to the web because it's such a limp platform.

And btw, the point I am making is not against high level constructs. Once you have sockets you can build whatever you like. The problem is HTTP is the bottom level of the web. If you had proper facilities at the bottom you could have your HTTP as a library and I can have my real-time protocols. The web constrains engineers based on some academic's rationalistic ideas about optimisation.

Still waiting for your answer.

How exactly do you expect build any automated tooling around your ecosystem of raw socket applications?

For example: indexing

You're going to need something that solves the problems HATEOAS solves when it comes to multi-service coordination.

Who actually uses or needs this kind of tooling? What exatly is "multi-service coordination" and why should anyone non-academic building actual systems care what it means? HATEOAS doesn't solve the problem anyway. What happens when a field requires fixed-point arithmetic to two places? Your tools are going to get that wrong unless you start adding types. In the end you need humans to read proper documentation and implement carefully thought out solutions. This is the XML conceit all over again.

Interesting. But it has one flaw; how exactly would I do this

curl https://api.twilio.com/2010-04-01/Accounts -H "Accept: application/json

inside a browser on a normal GET request?

The short answer is that browser vendors need to get their act together. They've shirked this because no-one's demanded it.

Fortunately until they do, broadly, a browser user will always want the newest resource, and the type can usually be defaulted (you're using a browser, so you want HTML by default where available). Additionally, XHR and related technologies usually allow adding custom headers.

Why would you want to do that from a browser?

Ummm... why wouldn't you? Rich web clients and all that?

I thought he meant doing it manually. Using XHR is trivial isn't it?

One way I have solved this issue is to look for the Accept header. If no valid formatting options are available in the Accept header look for a query string parameter.

To me this is vital. Being able to explore an api within an browser makes it exponentially easier to understand and use.

A lot of good points there, but the part about detecting mobile devices is a bit short sighted.

Varying your content based on mobile User-Agent (or any User-Agent actually) renders public caches almost impossible to get right.

>> Varying your content based on mobile User-Agent (or any User-Agent actually) renders public caches almost impossible to get right.

Interesting. What would you recommend doing instead, supposing, for example, that you are meant to be returning a different set of items depending on a device identifier, and that you have to support 100s of devices? Should this be considered a different resource instead of a different representation?

Also, can you point me to a nice resource about public caches?

Thanks!

At your cache layer (e.g. Varnish), you would need some VCL to collapse all the hundreds of User-Agents options into something sane like "X-Device", feed this header to your backend, and vary the content based on that instead. This way you can cache just a few representations per resource (e.g., "Desktop", "Mobile", "Tablet", etc.) instead of several hundred. The problem, of course, is maintaing this list of User-Agents.

On his mobile example, it's much easier to simply be pragmatic and serve content for different devices at differents URIs/domains altogether. It doesn't break if you can follow some kind of convention, such that the URLs from one map directly to the other (for instance, mysite.com/news/article -> m.mysite.com/news/article) and then you issue redirects accordingly.

Thanks so much for saying what needed to be said :)!

David Zülke has a very good talk about REST that he's holding at conferences around the world regularly: http://www.slideshare.net/Wombert/designing-http-interfaces-....

Ah yes the conferences... When will the RMM (REST Maturity Model) certification, books and classes be available? It will be the CMMI + SWEBOK conferences all over again gangbusters! j/k. All good stuff it is just a good portion of it is to sell books/conferences and sometimes the good engineering parts are lost on the idea that in the end engineering is making things more simple not more complex.

It is a good presentation and hits on many good points but also makes the REST model a little too narrow for most client/consumer usage today easily.

This webinar by Brian Mulloy is a nice complement to the post and offers up some great design tips when it comes to designing a RESTful API...

http://blog.apigee.com/detail/slides_for_restful_api_design_...

For more resources on this topic, I have been leafing through O'Reilly's Hypermedia APIs book, it's been interesting.

http://shop.oreilly.com/product/0636920020530.do

He didn't go over the common example of paging by including

   nav { prev: '...', next: '....'}

in the response. But, the thing that always concerns me about this is that it requires the client to maintain state. If you consume such a web service in a web app, and the user hits "next", you'll have to have stored the next url somewhere.

If you want to show any content to the user, you'll always need to hold it in memory somewhere; that's inevitable. Storing an URL for an action is just part of the rest.

Personally, I'd just use a closure and bind it immediately to the event handler of the UI element.

Isn't the whole point of REST/Hypermedia that the client holds the state instead of the server?

If I want to raise my blood pressure into the 180's all I have to do is open a thread about web design "principles".

He has the perfect anti-example for HATEOAS right in there: financial transactions. You would never in 1 million years want to discover the API for such a thing by experimentation, because there are specific precision requirements.

Every time someone tries to explain the benefits of this nonsense to me it's either 1) something you could already do with sockets 20+ years ago, or 2) a poorly motivated academic idea that doesn't get me closer to implementing a correct system with good documentation.

Reply on news.ycombinator.com