Choosing an HTTP Status Code
racksburg.com
racksburg.com
* I don't consider 307 and 308 to be irrelevant. Even if many browsers today are safe against method-change attacks, it's always better to be explicit.
* I don't see why 304 would only be used if "implementing a web server". Many web applications do their own Last-Modified or E-Tag checking behind the web server (and for good reason).
* There are also non-webserver reasons you'd use 408 and 413.
Also
> I don’t find this argument compelling, if for no other
> reason than this: in a world where everyone is moving
> to HTTPS, we’ve forbidden any proxy or caching nodes
> that are not under direct control of the server.
Otherwise I'm glad to see some people still think using a variety of HTTP status codes appropriately is a good idea. It makes services easier to use and more intuitive.
99.9% of the time use a 301 redirect.
https://github.com/for-GET/know-your-http-well/blob/master/s...
the user agent MUST NOT change the request method if it performs
an automatic redirection to that URI
Most browsers mis-implement 302; hence 303 (must change to GET) and 307 (must persist request method) were added to make the options explicit.
Of course, just because a standard now exists....
EDIT: Here's the rationale (from [2]):
"I think the underlying principle of the matter is that if the client is going to resend the POST request body data, then RFC 2616 wants clients to get the user's consent. The trouble is that most client implementations did not resend the POST data when following 301 or 302 redirects. So the principle of getting user consent doesn't really apply.
RFC 7231 (HTTP/1.1 June 2014) changes this. It ratifies the common behavior of 301 and 302 (to not re-POST), and in doing so it leaves it up to the client to determine when it is safe to automatically redirect."
1. https://github.com/golang/go/blob/master/src/net/http/client...
2. https://github.com/golang/go/issues/9348#issuecomment-867903...
As bandwidth to even home users go up, the need for local caching proxies might go down -- but at the same time, Netflix is an example of how the current architecture of the Internet isn't enough if you want to do individual media streaming (they maintain their own caches, but that just means that if one wanted to distribute a few TB of hd video, say an independent documentary series, or even a video diary, one might expect to either have to pay a high bandwidth bill, or do some "proximity caching"). Alternatives exist of course, such as using a torrent for distribution -- but throwing out all ideas of content proxying "beacause TLS" seems misguided.
I've created and used many a RESTful API and can vouch the ones that just return 200 even in cases of error are a big pain in the neck.
This is usually done (IMO) as a shortcut on the hand of the service provider, lack of knowledge on the conventional codes that should be used, or for the sake of more flexibility when endpoints change.
However, all of those reasons just push more work onto the service consumers. Lack of standardized status codes means the consumer must have customized error detection, which is more susceptible to changes as the provider pushes out updates.
Apache httpclient is a very well constructed library. And there are bunch of other smaller ones.
Mainly because I got tired of seeing certain classes of errors showing up in my main error logs, when they are being caught, but throwing 500 errors... mainly pen-test/bots trying to get in.
> I’m not completely sure they do matter.
> There’s a lot of smart people at Facebook and they
> built an API that only ever returns 200.
Most (many?) APIs, whether or not their authors call them REST, are single URL APIs that you throw JSON/YAML/whatever at. If the caller is going to get back a serialized status message (["Not found"]), then the HTTP request was successful, and it should return 200.
I'm not sure why people insist on spreading their APIs out between the transport and content layer.
That said, "single URL APIs that you throw JSON/YAML/whatever at" are not RESTful anyway. For a non-REST API, then it may make sense to return 200 even if something was not found, but it is still not a good idea to just ignore the properties of the upper-level protocol your data is ferried on.
> I'm not sure why people insist on spreading their APIs out between the transport and content layer.
* Browsers can't speak TCP natively (oddly enough)
* HTTP works through proxies
* Most web frameworks do HTTP more easily, same for web servers
Of course, the real solution here is probably to use websockets or HTTP/2, since then you get all the advantages of HTTP without the downsides of TCP.
That's by design - allowing arbitrary TCP in a browser is just asking for security and DDoS problems.
> HTTP works through proxies
So does TCP (SOCKS).
> Web servers
Using web server wouldn't make sense if you were using straight TCP.
The real reasons HTTP is used when plain TCP would be more appropriate are:
1) Lazy/stupid firewalls configured to only allow port 80/443. This leads to everyone doing everything over HTTP, moving the problem and making the firewall much less useful.
2) NAT. When the primary benefit of the internet - where each peer is equivalent in the protocol - cannot be counted on, centralized servers (usually using HTTP because of #1) are used instead of direct TCP.
A common solution is to make your public API RESTful and authenticated differently from your browser cookie sessions, and make the private web APIs always return 200.
Here the 403 is appropriate because the user is logged in and trusted to some degree by your system, but isn't allowed to access that URI.
If the user were unauthenticated and tried to access the same URL, he should get a 401 for Unauthorized, which is the same response he should get for every URI in your system, thus exposing nothing about your underlying service.
This behavior is explicitly permitted by the standard, FWIW: An origin server that wishes to "hide" the current existence of a forbidden target resource MAY instead respond with a status code of 404 (Not Found).
See https://news.ycombinator.com/item?id=276687 for further explanation.
So if your usecases would benefits from caching, discoverability... then you may want to choose REST.
In that respect, REST is a lot like relational calculus: certainly not a panacea, certainly not universally-applicable, but an extremely useful way to approach a large number of different problems. And just as one should have to justify why one is not using an RDBMS (and there are a huge number of reasons not to!), one should need to think hard before abandoning REST for RPC.
> If the API is indeed RESTful, and it complies with the
> concept that a URI represents an address of a specific
> resource
More seriously, almost every API I've seen that tries to be (or claims to be) RESTFUL is just somewhat randomly distributing parameters between the URI, headers, and a post body.
> HTTP is not the transport layer. It's an application protocol
> meant to convey application data
I imagine that always returning 200 drastically cuts down on support costs and basically-irrelevant forum posts, regardless of the technical rights and wrongs of doing so.
this is an argument for not using 200 for all; not against
>this is an argument for not using 200 for all; not against
The practical side of me disagrees.
Experience bears out that plenty of developers are at a loss when e.g. a login callback page shows an unhandled exception error despite error-checking code being in place and running against the response's "error" node or whatever is used.
The problem is that there just aren't enough people out there that can understand the entire stack, and the end result is that people who've paid (good/bad) money for development - and their end users - end up with services that'd otherwise be decently functional showing the odd exception error instead.
(the technical side of me agrees with you absolutely!)
Now all clients have to implement their own error checks, string parsing, just to discover something that the HTTP layer could have told you explicitly, that the resource requested could not be found.
If an API treats data as resources, then the API should return HTTP status codes that clarifies how the query against the resource was handled.
"Could I have this?" = "404 Not Found" is a great response.
"Could I have this?" = "200 OK" (not really) is a crap response.
Another bugbear is a server suddenly switching content types for errors. If you're on IIS, make sure you're handling your own damn error pages so I don't suddenly have to account for an HTML 500 error page in my javascript code. Cisco's AXL API was a terrible offender for this one.
My favourite are the "200 OK" responses with a Java error trace in the body of the response.
IMO HTTP status codes should return information about the HTTP transaction, and only about the HTTP transaction.
If you run an API layer on top of that, that's a separate concern.
So "Server busy" should not be in the same space as "That API request was bad."
On my last project I returned 200 for successful transactions, 404 for missing pages, and 444/closed connection for 'Piss off and stop making inept attempts to hack this server, use it as a free proxy, or rack up your SEO stats."
IPs trying any of the latter are blocked by Fail2Ban.
444 seems to have cut down on spam requests from botnets, and it's cheaper than serving a full 404 page.
* Make request * Has error status code? * Error
* Make request * Has error string message? * Error
Fairly similar code-paths. If there was a very consistent implementation of REST API's and how they worked/responded this argument might have more legs; because client libraries could have a shared base libraries which handled this logic. As it is, API specific libraries have to implement error handling logic so it doesn't really matter.
Additionally too many clients think that error messages are safe to be shown to end users. In reality error messages are seldom to never fit to be shown to end users, not least because they're not localised, but also because the target audience for an error message for an API is the developer, not the end user.
i.e. is the client a mobile one? Does it have limited space to display a message? Is the client actually a desktop client calling the API with specialist needs to show a system dialog with specific properties filled that is more than just a single error string?
An API is an interface to interact with the web application, beyond providing the means to interact with resources it should make few to no assumptions about how clients of the API will act, their environment or constraints, their locale requirements, or anything.
The client is best placed to decide how to communicate to the end user, and the best thing the API can do is provide clean and unequivocal instruction to the client that something has happened so that the client can do it's job of handling it how it feels is best.
Having spent three years of my life writing tooling to deal with such a vendor's API, I would really, really prefer that people avoid this anti-pattern. REST is a really nice idea, and it works out tremendously well in practice. RPC-over-HTTP is not such a nice idea, and it works out rather painfully in practice.
As an aside, the mere fact that the message is serialised doesn't mean that HTTP and rest are inapplicable: HTML, XML, JSON, YAML, protobufs, s-expressions &c. are all just different forms of serialisation which can be applied to data. The RESTful way to indicate Not Found is to return a 404 status code, with a body which is useful to the client in some way. A human being might accept plain text or HTML; a programmatic client might accept a JSON or protobuf response, and probably doesn't even need one for a 404.
It's not so much the nature of HTTP as the nature of RPC. RPC is all about calling remote procedures (hence the name); the issue is that there's almost always some internal state which is being mutated. REST makes clear what that state is, and provides reasonably clean semantics for idempotent updates; RPC tends to obscure that state. RPC can certainly be done right (after all, REST can be thought of as a constrained form of RPC…), but in many cases REST is the right level of abstraction to work at.
This is actual code drawn out as a state diagram. It's officially available for Erlang and Ruby. (Disclaimer: I am one of the "maintainers" of the Ruby version)
Are you a Teapot? Yes: Error 418.
* an old PoC in NodeJS exists that reads the states/transitions from a JSON file and calls the correct callbacks in the correct order. Implement the callbacks (semantics) and you're done.
Disclaimer: I'm the author.
What about a request for non-existing resource (a file)? That's a 200 OK with an HTML for you! Also, the content of the HTML is "<h1>200 OK</h1>".
Luckily most of the functionality in their products is not implemented through the APIs...
There's a company I work with that is similar to this: Requests are XML, successful responses are returned as JSON with status 200 (with an empty error status field), errors are returned as XML with status 200, and the error text node is always "An error occured [sic] with your request".
More humourous alternative: http://httpstatusdogs.com/
I completely disagree with this part of the diagram:
If the resource is secret then the entire guessable path should return a 401 or a 403, not a 404. Knowing whether something exists is a privilege that a user needs to authenticate in order to gain. This isn't just an academic distinction. Github does 404s for unauthenticated requests and python's default http client (at least for python 2) doesn't send credentials unless it hits a 403 or a 401. Since github sends a 404 you have to hack around the language.
404 means Not Found. But that isn't true. The real status code is Forbidden and it should be used on all possible urls that follow a guessable pattern.
For example, on Github we know that a user is at github.com/username, we also know that whether or not a user exists is public information. So if we GET github.com/nonexistantuser, that should return a 404.
A user's repo, however is not public information, so when a request is made to github.com/username/nonexistantrepo that should return a 403, unless you have privileges to know that information, if you were, for example that logged in user, only THEN should you give the 404.
Also it makes more sense and more simple to me that something I don't know about simply doesn't exists for me. :)
* 2xx - We've got it!
* 3xx - They've got it
* 4xx - You fucked up
* 5xx - We fucked up
Its a version 4 currently
I'd buy that for a dollar.
I've gone down the path of trying to shoe horn every error condition into a separate HTTP status code and (IMHO) it's a Big Mistake [tm]. You really are confusing transport and application layers and the transport layer simply doesn't have enough codes for what you need.
When something obviously fits into an HTTP status code then sure, use it, particularly when it'll mostly make your client do something sensible.
But you just can't get away from needing error codes that go beyond HTTP status codes for any moderately complex Web application.
"the existing status codes are much too general for a modern website/API... why worry about spending any time on a redundant, not-as-useful HTTP status code?
When pressed for a reason why it is important to use specific status codes, a common reason cited is that HTTP is a layered system and that any proxy, cache, or HTTP library sitting between the client and server will work better when the response code is meaningful. I don’t find this argument compelling, if for no other reason than this: in a world where everyone is moving to HTTPS, we’ve forbidden any proxy or caching nodes that are not under direct control of the server."
That said, Michael's counter-argument is well worth consideration. I've generally left HTTP status codes and Application status codes completely separate for any sizable project, but I find this article and argument worthwhile and will continue to reconsider, as I generally do upon every new project, whether using HTTP status codes to represent general application state is worthwhile.
The original point regarding complexity is most important to me, but Michael's diagrams do an excellent job of showing how Application state can be considered an extension of the Protocol in specific cases.
http://www.vinaysahni.com/best-practices-for-a-pragmatic-res...
It has a section about status-codes: http://www.vinaysahni.com/best-practices-for-a-pragmatic-res...
e.g. I return 304 when people query an image 1) Because it is not a static resource but is pulled in from another server and 2) It will never expire because if it changed, a parameter in the URL would also change.
This means that I can't ever have a 403. All I can return is a 401 or 404 or a 200 with an empty list. I kept fretting about this. Now I feel better. #how-i-stopped-worrying-and-learned-to-love-the-atomic-codes.
I tend to agree with that and even more so looking at these flow charts. Complex error handling is an easy source of bugs. At the very least, return codes should be limited to the minimum information needed to prompt the desired action in browsers, proxies, and search engines.
It looks something like this:
HTTP 403 { errorCode: 403102, message: "Missing required permission: IN_SOCIAL_NETWORK", id: "064d6a1f-ba30-47de-8975-c319f01729ca" }
This perfectly sums up what I've tried to say about this "robustness principle" for years. Fantastic.
SIP inherits these status code ideas, plus adds super-neato whiz-bang headers, like "Retry-After", that allow comments. Their actual example is "Retry-After: 300 (in a meeting)". It's beyond delusional.
Except.... someone read the spec. They see Retry-After. They then implement it in some big iron that connects to the PSTN. Little SIP VoIP provider comes along, passes along this dumb header, Retry-After. Suddenly, big switch guy is pissed, as Verizon has disabled all his lines as "Retry-After" gets translated into "shutdown trunk" or something. Awesome. Really a round of applause for the IETF there. (You have got to know that these headers came about just sitting around thinking shit up, vs actually trying to implement telephony. That's why SIP says it might be used "to run a chess game", but has a mandatory "Call-ID" field. Oops.)
It seems like such a common requirement, I'm surprised there's no agreed standard.
It renders simple middleware logic like "retry on a 5xx, don't retry on a 4xx" invalid.
If you'd call it "Server Over Capacity" you could just as well argue for a 5xx code.
Probably it's a thin line between client problem and server problem.
Your simple middleware logic is going to have to examine the 400 error in more detail in order to adjust its request rate e.g. exponential back off.
It's not that the "server is over capacity", its that the specific client is making "too many requests".
429 = Too many requests (rate limited)
Generally, the successful team will try to make their interface as usable as possible to that 'audience' (considering diminishing returns).
4XX response > Is the user being throttled? > Are you Twitter? > 420 Enhance your calm.
> Coda: On Why Status Codes Matter
> I’m not completely sure they do matter.
> There’s a lot of smart people at Facebook and they built an API that only ever returns 200.