Using HTML as the Media Type for your API | Hacker News Reader

Using HTML as the Media Type for your API

codeartisan.blogspot.co.uk

36 pointsDougWebb18 comments

I think one of the big reasons that JSON took off is because it removed the attribute/value dilemma, e.g. should I do this:

  <color pattern="checked">blue</color>

or this:

  <color>
    <pattern>checked</pattern>
    <value>blue</value>
  </color>

or even this:

  <color value="blue" pattern="checked" />

And lots of variations. JSON simplified everything into just maps and arrays, so there's one straightforward, simple way to do it.

Using HTML for your API response re-introduces all the confusion as to what to put where, and now you've got to decide which random HTML tag to use for which element. For 99% of the cases, I see no benefit to it.

The only thing I think of is if you're returning a very large and complex data set, and you want to take advantage of jQuery/similar to use selectors to extract certain patterns of data, which could be a cool thing do to. Have to say, I've never needed that before, though.

JSON wins because it's easier to manipulate JSON data directly in your language of choice because the data is already in a usable structure. If you instead receive data as HTML, you have to read that data into some native structure before you can work with it, which is too much hassle just for the minor benefit of having the data be directly presentable in a web browser.

... and by the way what if like most web designers I don't agree with the HTML layout of the data? For example I don't want list items in a bullet list, I want them in sortable table or in a combobox droplist instead. Then I have to convert the data into some readable structure so I can change it's layout, which means reading the data into a Javascript object perhaps ... gee, I wish the API just gave me JSON output in the first place.

"... which is too much hassle just for the minor benefit of having the data be directly presentable in a web browser."

The point is that this is not a minor benefit. You literally have documentation for your API built in. You don't need to write a whole set of docs listing what parameters are acceptable for which requests. You get the semantics of form handling and link-following for free. Having worked on an API that basically brings HTML form handling to raw JSON and XML (Twilio's), where we had to work hard to create documentation for what we were doing, I can tell you this is a big deal.

Incidentally, using a media type like HTML is the only way to build a true REST API. If more people understood that, we might be able to have a saner discussion around REST and its trade offs.

>Incidentally, using a media type like HTML is the only way to build a true REST API.

That would definitely be a worthwhile point to consider, if REST didn't have several completely impractical requirements that people inevitably abandon the moment they try to make their site do anything more complex than (unauthenticated) CRUD.

I don't disagree, I'm just saying that if we had discussions around REST that started with the assumption that you had to use HTML or something like it, then a lot more people would have a better idea of what REST actually is and could start to make more reasoned arguments about its costs and benefits.

Maybe so, maybe so. But I seriously doubt that that's the reason REST advocates have a hard time explaining the benefits of being unable to use authentication (and attendant sessioning) under a literal reading of "no out of band info".

I'd have to say that the major impediment in that case is that there are no benefits.

fwiw hal+json has documentation built in (arguably better than HTML does) and its browsable, here's an example API:

http://haltalk.herokuapp.com/

Just how much would this be different if the data was json?

   $("#listItems li", apiResponse).each(function() {
      $("#mytable").append("<tr><td>$(this).html()</td></tr>");
   });

Json or html, you're going to have to navigate to the list in the api response and loop over the list items to generate the markup you want.

Well, for a start, you've assumed the presence of jQuery and the DOM. DOM parsing can be a pain to do server-side.

The article points points out that DOM traversal is annoying, but jQuery-like apis are available for most languages.

Last paragraph... hammer, meet nail.

If you're just going to send plain HTML over the wire, why are you going to even bother thinking at the level of abstraction that most developers (well, at least me) tend to think of when "developing an API"? I'll give you that it's a loose notion, but there's another issue. If you're building a non-trivial web application, how does sending HTML as a media type help you to decouple your data from the DOM? I believe the answer is that it doesn't. You could make the argument that decoupling becomes less of a necessity when you are providing more "fine-grained" access by controlling exactly what HTML is generated from the server at each endpoint. I say it could be argued, but I wouldn't argue it.

I think it's somewhat of a benefit to make the clients expect to do a little searching to find the data/links/forms they want to use--it makes them a little more robust and adaptable (I guess because they have lowered expectations!). Contrast this to a client dependent on an XML or JSON Schema. Yeah, that client is easy to write, but you've now locked the server into not changing its format or you've doomed the client to break when the server decides to change anyway. I'm shooting for something a little better.

You can't totally decouple the clients; they have to understand in some way the vocabulary the server is using, but you can do a lot to decouple them from the structure of the markup.

I think the thing that keeps people from adopting HTML as a content type for APIs is the fact that tools don't parse HTML into native data structures. All the tools build up a DOM structure, which makes sense since that's what HTML is generally used to represent.

If we could agree to use some subset of HTML that would map to native objects in most languages, then we could build tools around that standard.

Or we could approach from the other side and just try to bolt on some hypermedia features to JSON, as projects like HAL are attempting to do. This seems to have more traction at the moment.

Yep, this is really the point. If we think hypermedia APIs give you evolvability, then we've got to figure out how to do it; we don't have libraries for that sitting around, so someone has to do some work somewhere.

HAL is going for "start with something clean for machines (JSON and XML), layer in hypermedia, get people to adopt it." This seems like a perfectly reasonable approach.

I'm suggesting another approach, which is "start with something that supports hypermedia and is already ubiquitous on the web, and figure out how to get machines to make sense of it."

I'm not sure either one is easy, so it's worth having multiple horses in the race, I think.

If you are concerned with having your API conform to HATEOAS, HTML offers some clear advantages over JSON. Namely, HTML has well defined "hypermedia affordances", or "H Factors"; plain old JSON has none:

H Factor

http://www.amundsen.com/hypermedia/hfactor/

Hypermedia-Oriented Design

http://amundsen.com/articles/hypermedia-oriented-design/

It's possible to develop and document a well-defined "semantic profile" for a hypermedia API that uses HTML as the base media type. For an example see:

ALPS - Application-Level Profile Semantics

http://amundsen.com/hypermedia/profiles/

By leveraging some or all of the details of the API, machine-to-machine logic can be developed fairly easily. In his recent book Building Hypermedia APIs with HTML5 and Node, Mike Amundsen demonstrates an example of a "microblogging bot" that consumes an API conforming to the semantic profile detailed in the ALPS document. The bot (written in JavaScript), knowing only the semantics involved with registration and "create post" links/forms, and an entry point for the API, is able to signup and begin making posts.

The interesting thing is that the precise implementation of the API is not important, e.g. whether it's written with NodeJS or PHP or Clojure or whatever. Also, from one implementation to another, the pages served by the API could vary in terms of the layout of the pages, CSS, etc. So long as the pages conform to the semantic profile, the bot should be able to work equally well with a wide variety of implementations.

Regarding JSON, there are a number of attempts to develop hypermedia-rich media types on top of it:

HAL

http://tools.ietf.org/html/draft-kelly-json-hal-03

JSON-LD

http://json-ld.org/

Collection+JSON

http://www.amundsen.com/media-types/collection/

Siren

https://github.com/kevinswiber/siren

There are others as well.

A lot of Mike Amundsen's work you've listed here is a direct inspiration for this article. Thanks, MCA!

FWIW, with RDF (RDFa, etc) and such, you could have machine readable and human readable API endpoints. Kind of like XSL but a decade or more newer, and possibly inclusive of it, depending on what you want to do.

Reply on news.ycombinator.com