Hacker News Reader

Comment by larsnystrom | Hacker News Reader

Good article, but this not a “trick”, it’s a core part of the HTTP protocol. Worthy of an article non the less, judging by how misunderstood the topic is among the commenters here.

View on HN

Content negotiation itself is not a trick, but this usage of it is fair to characterize as a sort of trick. Most uses of content negotiation are about serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document. The fact that it works is unexpected to the uninitialized, because even just reading the RFC will not necessarily make it apparent why cURL and the browser would return such different results. There's a lot of ways a server could detect cURL, like the user agent, and it might come as a surprise to some that cURL and other user agents that are not web browsers often send */* for the accept header. It is not a terribly new way to use content negotiation, but arguably still a clever one. It's a deliberate behavior.

> serving the same content in different formats, like different image formats; in this case though, it's actually negotiating a different thing entirely, where one is a machine-readable CSV and the other a human-readable hypertext document

Huh? No, it's also serving the same content, just in different formats: one is a machine-readable CSV and the other a human-readable hypertext document.

Disagree: the table inside the HTML is the "content" that comprises the entirety of the CSV. Related but absolutely not "same". (It'd be more arguable if all the HTML had was the table, but it's actually just a normal web page with a table.)

Well, neither a picture is really "the same" if it's encoded in different formats, say, JPEG or PNG.

Actually, that's a pretty good point. The thing is that the URL itself refers to some conceptual resource, and the response is ideally a representation of that resource, potentially one of multiple. If you take the same source image and encode it multiple ways, although the two resulting images are different from eachother, they are representations of the same underlying image/resource. But if you were to provide a different image, or alter the image in other ways, I think this would be pretty tricky actually, even if the modification was something trivial. You can imagine a simple use case like a WebP image with the text "Your browser supports WebP" and a PNG image with the text "Your browser does not support WebP." The point is that content negotiation being used to present different logical resources that are not necessarily interchangable representations of the same data feels tricky. I think that is still compatible with the fact that it's all within specifications.

Arguably the resource is the dataset of stock exchanges, and the CSV representation is forced to omit all the metadata but the HTML representation isn't.

I understand what people are getting at, as it's not really that big of a logical leap. I think the fact that it is somewhat of a stretch, but still "in the lines," is exactly why it is a "trick": it isn't doing anything particularly invalid or hacky, it's just not necessarily what you'd imagine when reading the RFC. Content negotiation to me is more about serving the optimal content to a given agent, not really about selecting modalities for different use cases based on different types of user agents.

I think both cases are "valid" although I think it is inherently less tricky if the document talking about and previewing the dataset is referenced via a separate URL from the dataset itself. (Which, of course, entirely mitigates problems like Apache Spark having HTML in the Accept header.)

This. lol. I opened the article and was ready for a cool trick and then they showed it returning csv over Curl and html over a browser and I was like "So the accept header? Maybe there is another trick...nope, just the accept header." Our API already does this at my company, where it returns an HTML formatted, human readable response over a browser nav but returns application/json over anything requesting only that.

I didn't realize you could assign a preference hierarchy... to me that's the trick (well, that browsers do it, anyway)

My impression is that many power users (not web developers) want to download datasets. Sometimes they’ll use a browser to grab a csv or json file, then stuff it into excel/R/python/… to do some sort of analysis work. This kind of knowledge is helpful for that crowd. They often don’t know about the non-url parts of a request.

Yeah, asp.net for example, supports both JSON and XML and relies on the accept header to decide what to send.

I was expecting something less obvious other than "101 class on the HTTP Accept Header" lol.

You would be surprised how little people understand HTTP despite working with it for years.

I’m just happy they didn’t call it “a hack”.

Reply on news.ycombinator.com