Your Web Service Might Not Be RESTful If… | Hacker News Reader

Your Web Service Might Not Be RESTful If…

theamazingrando.com

82 pointspsadauskas59 comments

Fortunately, I don't care.

The broad concepts of REST are good, but this pedantic stuff is so silly. There's a reason that most popular APIs will fail this test -- the people who built them were thinking about more important issues than achieving some kind of religious purity.

I agree. For most web services, the REST purity test is unnecessary.

My only point of agreement with the author is that web services that aren't RESTful shouldn't describe themselves as RESTful.

Most APIs are not restful because their designers had more important issues - Would you say the same for most web pages breaking web stanards or is that another matter entirely?

It's more like using tables instead of CSS -- the only real compliance with web standards here is valid vs. invalid use of HTTP; everything layered on top of that is mostly subjective in the case of REST dogma. In fact, some of the things in this article directly contradict recommendations in O'Reilly's RESTful Web Services.

First author of HTTP 1.1 is Roy Fielding, same guy who defined REST. In fact, http 1.1 was designed largely to make HTTP restful. As such, you'd be surprise how far HTTP goes towards RESTfulness. For instance the spec defines idempotency and satfety of methods. Ergo, servers may produce valid requests and responses but still fail when it comes to being valid http if their internal operation does not conform to the operation semantics. Look up the google web accelerator fiasco to see how things can go terribly wrong.

To be clear, the Google Web Accelerator fiasco was not related to anything Google did wrong with Web Accelerator. It was a result of other people doing things wrong.

Certainly there's a reason, but that's not it. The reason is generally that they were doing what was most convenient at the moment. There are occasional cases where someone was actually thinking about some requirement that conflicted with the REST architectural style, or had an application where the benefits of REST didn't matter. But the much more common case is that someone just wasn't thinking about REST, usually because they don't know REST.

"There's a reason that most popular APIs will fail this test -- the people who built them were thinking about more important issues than achieving some kind of religious purity."

What issues are those? The article does give reasons for each recommendation that go beyond an appeal to RESTful Authority. I would be curious to hear how the criticized techniques solve some important issue better than the article's recommendation. For example, is there a specific reason that API tokens are better than Digest authentication?

Perhaps creating real business value.

So when API designers do something like define their own status codes and numeric error codes[1] instead of using the universally-known response codes built in to HTTP, is that solving some "more important issue" or is it just adding more work for clients because of a not-invented-here attitude?

1. http://posterous.com/api/reading

What's the HTTP error code for "address didn't match" or "over credit limit"?

(honest question, not snark) - How do you handle application specific messages and errors using the normal http error codes?

Not sure what you mean by "address didn't match" but it would likely be 404 or 403 depending on what the "address" is.

"Over credit limit" would be "402 Payment Required."

You can always return additional, application-specific details in the body of the error response. But choosing a correct HTTP status code is an important first step because it has well-defined semantics (e.g. caching behavior) for client libraries or proxy software that know nothing about your application, and it allows the client to use a single code-path for all error handling.

I was picking on the Posterous API because each response has up to three "statuses" - the HTTP status code, the rsp@stat attribute, and the rsp/err@code attribute. The docs don't even mention what combinations of HTTP status code and response body are valid. (Will errors be served as 200 OK? Will some responses have 4XX or 5XX status codes but not have an "err" element in the body?)

As it is, Posterous clients may need to handle a 200 response with an error in the body, an error response with an error in the body, and an error response without an error in the body (which is not documented in the spec but probably can't be ruled out). Not all client writers will think of all these cases, and not all will handle them the same. It would be better for the API to define its behavior in terms of HTTP, which would put all error handling in a single place.

and an error response without an error in the body (which is not documented in the spec but probably can't be ruled out).

Yes, it's probably more or less guaranteed that if you're hitting Posterous's API through proxies (your own local proxy; your ISP's transparent proxy, if you're not in the US and/or make a bad choice of ISP; the proxy in your company's firewall; a reverse proxy deployed on Posterous's side) that proxy will sometimes generate error messages without any knowledge of what Posterous may or may not have documented.

I haven't read Posterous's spec, maybe they have some way of preventing this, like requiring SSL.

Having a more specific error code/description in the body of the response is fine, it's using yours instead of HTTP's that's a problem. In the Posterous example that was linked, "Invalid site id" sounds a lot like 404, "User does not have access to this site" sounds a lot like 401 or 403, etc.

Hi, I'm the original author.

Those people weren't "thinking about more important issues", they were being lazy (or just copying what was being done already, without thinking about why). Its akin to javascript before ajax an jQuery, before we knew any better, we just copy and pasted code off the internet that worked, without spending the time to discover there might be a better way.

Because people are "thinking about more important issues", they're making it harder for consumers to write clients of their API, and they're presenting themselves with unmaintainable web services in the future.

In your article, you listed the following scenarios in which someone would be sad they did things the wrong way:

- Changing the URIs of some resources in order to load-balance across servers, which is a lot easier if clients get those URIs via hypertext instead of from the protocol spec.

- Screwing up user authentication when you could have just used digest authentication.

- Having your service run really slowly because you can't just use Varnish to cache the cacheable parts. (Maybe you didn't actually say this.)

I can't actually find any others in your article. This discussion thread has added a couple more:

- Getting HTTP error codes from an intermediate proxy server instead of from the intended origin server, which will make you wish you had used those HTTP error codes to indicate errors in your app, so that clients would handle them correctly. (There are a lot of ways proxy servers can get involved.)

- Having random robots (Google Web Accelerator, web crawlers, etc.) trash your database by GETting URLs that have destructive side effects.

- Having people not able to explore your API with their browser.

What are the other scenarios?

I really hate it when technical discussions devolve into accusations of "religion". You're basically saying that there is no possible rational reason to do things in some way; after that, any disagreement is merely apologetics. What's left to discuss?

Nice to see the way it's 'supposed' to be done, but really that's a lot more complicated than it needs to be. Most APIs can get away with "HTTP requests that do stuff".

As an example, here's how you create a new meeting with Twiddla's (evidently RESTless API). One POST over SSL:

  > POST https://www.twiddla.com/new.aspx
  >      username=billy
         password=iLikeRainbows
  
  < HTTPS/1.1 200 OK
  < 109232

Now, here's how this author suggests people do it:

  > GET http://api.twiddla.com/
  
  < [XML packet full of endpoints comes back]
  
  > GET [endpoint you parsed from that list]
  
  < HTTP/1.1 401 Authorization Required
  
  > GET [same endpoint after jumping thru crypto hoops]

  < HTTP/1.1 200 OK
  < 109232

As an API publisher, I can see absolutely no value in making my users jump through all those hoops just to use my API. The API exists so that people can get things accomplished on my site, and I don't see any reason why it shouldn't be just as usable as the site itself.

Please, if you're putting out a simple API, don't make it overly complicated just to please this author and others like him. Make it exactly as complicated as it needs to be to perform the task intended and no more.

Well he's just spelling out the steps to explain HTTP Digest.

In the real world, any HTTP client library worth using already implements HTTP authentication transparently to you, the API consumer.

That's actually surprising, that the default behavior for a HttpWebRequest.GetResponse() method would be to parse the response and decide to send off a second request automatically. I would expect some developer involvement would be required.

Regardless, assuming I'm using your library, I'd still need to do a request, some XML parsing to discover the right url, followed by a second request, just to get back the response I want. Compared to 1 request, that's still 3 times the effort for no added benefit.

It's an interesting artefact of the libraries we use that digest authentication is difficult, but you can "just use" SSL.

They make digest easy, too:

    curl --anyayth --user admin:sekret http://exmaple.com/resource_protected_by_digest_auth

Most libraries I've used have the option to respond to 401s automatically, and a lot of them have HTML parsing, too.

There are some added benefits. The load-balancing example he gives in the article is one; a second one is that there's less stuff to get wrong in the client code, generally. Instead of working from some informal equivalent of WSDL, you're writing code that interrogates the service at run-time to find the current definition of the interface.

If more people did proper restful applications, perhaps there would also be proper libraries for interacting with them? Perhaps something that looks like this (with a ruby flavor):

RestfulApi.new('http://api.twiddla.com/, :user => 'billy', :password => 'ILikeRainbows').post('Meetings')

Yes, exactly. I've written a (rough) DataMapper adapter for interfacing with them.

    class User
      include DataMapper::Resource
      resource_name "AllUsers"

      # define some properties
    end

    User.first(:login => "admin")

http://github.com/absperf/dm-ssbe-adapter/tree

"< [XML packet full of endpoints comes back]"

I'm going back and forth on this one. On the one hand, it is a pain to go through those steps if you already know which service you want. On the other hand, having that listing available makes the site kind of self documenting, in that you can explore the services available.

What I find more potentially useful is the idea of putting URLs into the XML for subordinate resources instead of some kind of ids or other identifiers. Then the client can follow those URLs to get a user's documents, or whatever it is you are modeling. Anything where there is a relationship but it is too expensive to just dump all the data in a single XML response.

It's pretty intuitive, and arguably easier than just supplying an id that the client must use to construct the appropriate URL.

Once you write a client, and its pretty simple, you don't have to worry about that step at all. And since the initial document rarely changes, and it can be cached, you usually only have to get it once. Here's some code in ruby that uses a service like this to find a user by their login:

    def find_user(login)                
      users.select { 
        |u| u["login"] == login 
      }.first
    end

    def users 
      users_href = services.select { 
        |s| s["name"] == "AllUsers" 
      }.first["resource_href"]
      JSON.parse(
        @http.resource(users_href).get(:accept => SSJ).body
      )["items"]                      
    end
  
    def services
      JSON.parse(
        @http.resource(@services_uri).get(:accept => SSJ).body
      )["items"]                   
    end

@http is a handle to an HTTP library, and @services_uri is the well-known URI. So its not nearly as complicated as it might seem at first, and the benefits are tremendous.

You're combining two issues. HTTP Digest requires the initial 401 unauthorized, but the same number of requests after that (And curl/libcurl handle it all for you, you don't need to think about it). I was just attempting to show what was going on, and why it was more secure than an API token. You're example requires that every API call use SSL, which means noone is allowed to cache it, which is going to be a hinderance on the scalability of you API (not impossible, though, but you're going to have to come up with your own, rather that just throwing squid in the middle).

The other part is discovering of the service locations. By not doing so, you're handcuffing yourself. In your API, you are stuck forever with `/new.aspx`. Should you decide someday to have something else you want clients to be able to create, you can't separate it out into `/new_foo.aspx` and `/new_bar.aspx`.

Doing these kinds of things actually make it less complicated in the long run, just like using jquery is less complicated than copy/pasting 50 lines of javascript of the web.

Hmm. You seem to have rebutted my argument by retyping your article.

The API in question has exactly one endpoint, so no possible value could come from having to look it up on a list each time you want it. And it is a POST to create something, which means you'd never cache it.

I picked that example because it is an existing use case that demonstrates that the rules you're proposing don't apply to every situation.

I must have missed the part where you said the API would only have one endpoint, ever. Roy's thoughts on the constraint is that there is only one well-known entry point, and my article was a little more lenient with "very few". Each one, however, is an additional maintenance burden on both the app writer and a client writer.

If your app only has a single resource, then there's no reason why it can't be the one well-known resource itself. There's no need to be snarky, these constraints apply perfectly to your application as it is. If you have any intentions of your app growing beyond that single resource, however, its not that complicated to add these constraints to make that future growth easier.

At the end of his article the author says, 'Now you don't have any excuses.' I respectfully beg to differ. I think the 'proper' way of doing things he describes is much more work for the clients of the API. The benefits to the clients he describes, such as not being mercy to changes in the API's URL structure, are indeed benefits, but they're not guaranteed to be realized, and it's questionable whether they would pay for the extra work required. Implementing the API as described would allow clients to create client apps more tolerant to changes in the API, but the additional work required to take advantage of it may be more of a barrier than a help. In a perfect world, the clients would have the time to do so, but in an imperfect one it probably just makes more sense for them to update their app when the API changes and munge some URLs with a shell script.

The author also conveniently ignores a glaring detail that always makes me angry when people talk about REST as something people should adopt. The simple fact is that browsers don't support REST — no browser supports the PUT or DELETE methods, not to mention other things in the stack like the web server. Yes, Rails has built-in support for REST, but it's accomplished via a disgusting hack whereby hidden form variables are sent to mimic non-supported verbs.

I've always felt that saying that people should adopt REST when browsers don't even support it is like saying people should switch to wind power when there are no wind farms producing it. Nice in theory, but not practical and far more trouble than it's worth.

Sorry, but browsers absolutely support arbitrary HTTP methods, and have since IE6. XMLHttpRequest, have you heard of it?

Also, REST != PUT + DELETE. Don't get hung up on it, they're just surface-level niceties. The real point is to not mix up GET & POST, to be careful about state and idempotence.

http://en.wikipedia.org/wiki/Representational_State_Transfer...

General problem is not getting the clients to change but get all the clients to change at same time. Or else you will end up hosting multiple versions of webservice. In enterprise setup its really hard as each organization can have different processes to be followed and value proposition for switching to new version of webservice may not be same for each client.

I agree, that REST is (slightly) less appealing if your only clients are browsers. However, my main gripe is with XML/JSON API's designed for clients other than browsers.

>> You have an “API/Key/Token” in a header or a url.

This - specifically, how to do user authentication/access control in a RESTful manner - has always puzzled me. The author suggests HTTP Digest authentication, but that seems to fly in the face Roy Fielding's commandment that a RESTful interface shouldn't depend on the communications protocol being used; HTTP Digest seems pretty tightly coupled to HTTP, no?

Broadly, what are some of the canonical ways to do user authentication while still keeping close to the theoretical/theological underpinnings of REST?

This is an interesting blog post on http authentication woes:

http://www.artima.com/weblogs/viewpost.jsp?thread=155252

Every single one of these concerns can be solved wuth digest auth over javascript: http://www.peej.co.uk/articles/http-auth-with-html-forms.htm... and http://www.berenddeboer.net/rest/authentication.html

RESTful interfaces should NOT be ptotocol independent. While REST can be implemented over multiple protocols, a RESTful interface is by definition tightly coupled to the protocol it goes through. For instance REST specifies that we should use a uniform interface, but HTTP tells us what that interface is, and a REST over HTTP interface (the most common RESTful interface) has to respect that. I would be interested in where you read this fielding quote as this 'protocol inependence' and its problems is a common charge he makes agains WS-* if anything.

http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hyperte...

Specifically:

A REST API should not be dependent on any single communication protocol, though its successful mapping to a given protocol may be dependent on the availability of metadata, choice of methods, etc. In general, any protocol element that uses a URI for identification must allow any URI scheme to be used for the sake of that identification. [Failure here implies that identification is not separated from interaction.]

Well, Roy has a way of describing things from a Systems Engineering perspective. that can confuse when read from people with a different context. I think where HTTP digest etc. comes in is the 'successful mapping' bit. While the abstract API should not specify HTTP digest, the materialised API that people actually use (the mapping of the API onto HTTP) should.

Amazon has clients sign the request: http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAu...

Good points. I think that people resort to URL based versioning and api keys in the url because they want to have it work from javascript/html, actionscript(flash)/silverlight clients without having to modify headers as much.

These points are great though and help make better restful apps.

If only someone built a nice, RESTful module for JavaScript, instead of these crappy (I'm looking at you, primarily, Prototype) AJAX interfaces.

Ponder... something to include in my in-progress library, perhaps?

Here you go: http://www.thoughtbot.com/projects/jester

Unfortunately, that's fairly heavy - and depends on Prototype for a lot of things (including class creation, etc). Unfortunately, Prototype's handling of object manipulation is extremely slow, especially when calling a base method: http://ajaxian.com/archives/javascript-inheritance-performan...

This is why I'm making this independent of other libraries. Implementing a nice interface to the mess that is XMLHttpRequest is not that difficult - and for what should be a tight, lightweight library, requiring something like Prototype is overkill.

What would a RESTful API for a queue resource look like? I'm thinking of something that supports push, pop, shift, unshift, etc.

Is there a meaningful difference between:

GET /user/1

GET /search?user=1

GET /?method=search&user=1

Is REST about the URLs you use, or the operations on those URLs, or both?

    GET /q

    200 OK
    { 'start': '/q/start', 'end': '/q/end', 'shift': '/q/shift', 'pop': '/q/pop' }

    POST /q/end
    x

    200 OK
    Location: /q/some-identifier-for-element-x

The queue now contains [x].

    POST /q/start
    a

    200 OK
    Location: /q/some-identifier-for-element-a

The queue now contains [a, x].

    POST /q/pop
    POST /q/shift

With obvious results. I'm not totally happy with the last 2 operations, another solution might be to GET /q/start or /q/end and DELETE the returned queue element URL. If the delete succeeds, do what you want with the data, otherwise assume that somebody got to it before you and attempt another GET-DELETE.

Neither method handles "client never got the server's response" very well, I'm not sure what a better solution would be.

Totally generic things like this are unusual, you can usually come up with a better set of resources and operations based on your problem domain.

REST is about the operations on the URLs; the URLs themselves should be totally opaque.

Without knowing anything about how the 3 urls you gave are used, what other operations are possible and how they interact with other resources it is impossible to say whether you're doing REST. The third one looks very suspicious, however (the second one too, though less so).

OK, so being perfectly RESTful is a pain. Just be RESTful enough.

Using custom XML is also not very REST. I would write an API in a way that uses the Accept header to demand either a HTML microformat or JSON, with the former laid out for web browser use.

Using custom XML is perfectly fine. It may not be the way you'd do it (or me for that matter), but there is nothing wrong with XML as a representation for data.

In fact, this is what XML was actually designed for.

XML is fine. Custom is the problem. A browser can't read it and the syntax is evident only by example.

Browsers aren't supposed to be able to read it. If someone is spitting out custom XML, the client is not meant to be a browser, it's meant to be an app of some sort, or an Ajax client who can read it.

If you're doing REST, browser-readability is good. It allows the API to be remotely explorable, self-explanatory, something a prospective user can test-drive with no investment in client code. Plus of course, it's a damn sight easier to test during development.

See, there's the problem. You're trying to wrap a custom data model around HTML. HTML should model one thing: HTML documents. Browser-readability should have nothing to do with a REST API to a web service.

But your browser is attempting to render an XML document, then the server probably has a MIME content-type error someplace.

I do agree that it is nice to be able to test something pretty easily, but the browser is probably the wrong thing to use for this. I always end up mocking up some scripts using curl or just manually typing in commands to the HTTP server using: telnet hostname 80

See, there's the problem. You're trying to wrap a custom data model around HTML. HTML should model one thing: HTML documents.

This is like saying, "See, there's the problem. You're trying to express mathematics with pencil marks. Pencils should write one thing: black marks on paper."

I don't see how "HTML documents" in any way constrains the semantic range of the thing being modeled.

Why is JSON ok, then? Its structure would be just as custom, and a browser can't read it either.

(also the entire world does not revolve around browsers)

JSON's syntax, at least, is defined. Also using JSON implies that extra stuff will be ignored and missing stuff rendered as null rather than causing the parser to crash - it's a forgiving format. Meanwhile it's a lot lighter than XML/HTML. And of course, I'm envisaging use of the "Accept" header to pick a format. Browsers ask for HTML. API users might ask for JSON.

I agree with all your points, but none of them have anything to do with how "REST" something is.

Reply on news.ycombinator.com