API Company Mistakes - Serialization Formats | Hacker News Reader

API Company Mistakes - Serialization Formats

rdegges.com

When developers ask for a serialization format other than what you're providing, it's generally because they're using a particular language that has a parser they like (or is the only parser they're aware of) for that particular serialization format. Moreover, the language they're using might be dictated by their employer and not by personal choice. A combination of those reasons is why SOAP and XML have lived on for so long.

What you should derive from this, however, is not that you need to provide better support for that particular serialization format; rather, if you want to make this particular set of developers happier, you need to provide better support for that language they're using. In general, this means building SDKs for your API in whatever language your developers prefer.

When given the choice between building an integration with an XML API from scratch, building an integration with JSON API from scratch, and building an integration with an SDK that does most of the work for you, developers will generally pick the last option, regardless of how the data is being serialized underneath.

> After thinking this through, however, I realized that one of the largest mistakes an API company can make is to remove previously advertised functionality (why would developers trust you to provide data to them if you're so quick to remove functionality?).

It seems to me that when providing a new feature in your API, you should have a latency period at the beginning where it is not guaranteed to continue to exist. Sort of the dual to the deprecation period at the end-of-life of a feature (would this make it an "imprecation period"?)

After that period passes, that's a signal to your API consumers that the feature will definitely be sticking around and receive long-term support. But until then, it could be removed without warning at any time; all code that relies on it should have alternate branches to handle the case where the feature endpoints pop out of existence.

Pre-1.0, all features are implicitly imprecatory, of course.

What kind of language and library makes creating an XML document is difficult? I know it's fashionable to hate XML, but seriously, it's a perfectly fine format, especially for serializing simple documents. The closing tags add some overhead; big deal.

I'm not even sure why much of a format was needed at all for this scenario. According to the docs[1], cURL will get a plaintext string, which has to be sufficient for nearly every implementation needing CNAM data. I'm quite sure I created a similar CNAM API in 2006: HTTP GET, return 200 OK and a string, or an HTTP error. Other major vendors' APIs are not difficult to work with, either (although contracts, VPNs, etc. can be annoying).

One reason there are more complicated APIs is for performance. While HTTP is simple, pipelining leaves a lot to be desired, and there is a benefit to being able to just use UDP.

As far as the data returned: I'm not even sure what an application supposed to do with a relative URI for a number after querying for that number? Or the price, which is apparently fixed per account? I cannot fathom why something as complicated as YAML would be needed to deal with such a simple situation: I'd wager customers asking for that are trying to be trendy or something, especially as JSON is a subset of YAML.

Not to be too negative -- OpenCNAM looks like the kind of thing telephony needs: easy, online signup, straightforward pricing.

1: https://www.opencnam.com/docs/v2/quickstart

XML is a mediocre format (the problem it solves is not hard, and it does not solve the problem well), but more importantly in this case, it's a format for serializing documents, and OP was trying to serialize objects. Obviously, we needs to add some core to perform the translation, though with any decent library it shouldn't take a full day.

> What kind of language and library makes creating an XML document is difficult?

Note that everything is relative. Taking Python, serializing any combination of "core" data structures with json is a single call (json.dump/json.dumps). XML? Well I could use XML-RPC serialization but nobody does that anymore, and so I've got to define my own serialization format with all the right conversions and shit.

XML is a mistake. There's rarely a practical advantage to a novel XML schema. If you're not using a well known schema (e.g., Atom) then omit XML support.

Not really. Imagine if HTML was implemented as JSON. Consider a hyperlink:

  { 'a' : 
    { 
     'attributes' : { 'href' : 'http://www.google.com/' },
     'val' : 'Google Search Engine' 
    }
  }

That's insanity when compared to an XML format:

  <a href="http://www.google.com/">Google Search Engine</a>

XML is actually a great hypermedia format. It just got a bad wrap with SOAP, WSDL, etc.

A JSON link format more similar to the given XML would probably be:

    {
      'tag': 'a',
      'href': 'http://www.google.com',
      'val': 'Google Search Engine'
    }

But you're right — if HTML had been based on JSON it would've been a nightmare to write. JSON is a great general-purpose format, but XML beats it out for constructing hypermedia. Not in all cases, though; compare the following:

    ['Peter', 'Paul', 'Mary']

    <ul>
      <li>Peter</li>
      <li>Paul</li>
      <li>Mary</li>
    </ul>

So basically, it just depends on what kind of data you're transmitting. :)

The ul > li example would quickly become a nightmare if you had to specify a CSS class or start nesting content in an li.

Indeed. XML is a bear to work with and the complexity adds little value.

YAML is just annoying and requires extensive documentation. I still can't figure out why it's popular in the Ruby community as opposed to much clearer Ruby files.

JSON FTW. The only major issue is multiline support, but overall it's a readability win anyhow.

I disagree. XML has a huge advantage over JSON: it can be streamed.

When working with JSON, you must load the whole JSON document before you can start processing it. If the JSON gets big enough, you start having performance / memory issues. One might argue that you can just get more powerful hardware, but that's only half of the equation: you'd solve your problem, but not that of your clients. Are you going to tell every single one of them that they just need to spend more money on hardware?

When working with XML, you can choose to store the whole document in memory, but it's merely convenient, not compulsory. Using a SAX-like parser, you can stream the whole process and get a much smaller memory footprint.

Having never worked with YAML, I don't know whether the same argument applies.

This is more of a personal issue than an actual one, but another problem I have with JSON is that once you support it, people are going to demand you support JSONp - terribly convenient, but a major security hole.

Aside from that, JSON is obviously a great format for small, self-contained responses that can be consumed by both browsers and heavy applications. It just doesn't scale well with large responses.

Is there any fundamental reason JSON can't be parsed in a streaming fashion? It seems like you could implement the same kind of one-token-at-a-time parsing you can do with XML. What am I missing?

JSONp is only a security hole for the client, not the API provider. I suppose it puts a bit more pressure on the API provider to not get compromised, though.

Well, I thought it wasn't possible to parse JSON in a streaming fashion, but given the number of annoyed answers I got, that was obviously incorrect.

Let me amend that to the more qualified "the libraries I've used so far don't allow you to stream JSON". My point is still somewhat valid - most XML libraries I know offer stream-based parsing out of the box, most JSON libraries I know don't - but it's not the show stopper I made it out to be, since it turns out you can, after all, implement your own streamed based parser.

As for JSONp, you're obviously and entirely right. My point was that, in case something happens on the client side (through an entirely different security hole - and let's face it, people who feel comfortable consuming JSONp are likely to take other insecure shortcuts), I'd much rather not be implicated at all than have to prove I had nothing to do with it.

JSON is just a streamable as XML, if not more so.

Customers demanding JSONp is not a good reason to not use JSON.

So having dismissed your two points, I think it's safe to say XML does not have any huge advantage over JSON. Certainly minor advantages, though those are of mixed value and often disadvantages.

While in theory JSON might be just as streamable as XML, in practice I find every XML parser I have ever used supports streaming, and I have never seen a streaming JSON parser.

Actually, having just googled I find a couple, but the only C one I can find (yajl) seems to be mostly abandoned, with a huge number of open issues.

> When working with JSON, you must load the whole JSON document before you can start processing it.

This is not the case; JSON is every bit as amenable to streaming as XML, in my experience. Fewer implementations may support that, of course, but that's not really the fault of JSON. The Jackson parser for Java, for example, has a complete streaming API.

The API should be independent of any serialization format. The serialization layer should take care of it.

Reply on news.ycombinator.com