The rise and rise of JSON (2017) | Hacker News Reader

The rise and rise of JSON (2017)

twobithistory.org

83 pointshenridf106 comments

I write a lot of smaller service agents, connecting legacy systems or feeding them with data in the public sector of Denmark. Most of those are done with XML and it’s often quite terrible.

We write a lot of C# but one of our tools is an old adobe lifecycle server, and .Net XML isn’t the same as Adobe XML. I mean, technically it’s just XML, but the two techs expect your XML elements to be build a certain way and break when they aren’t. So in order to make the two techs talk with each other through SOAP, we had to write a custom library to translate the .Net xml before we send it. And that’s just one in a long line of similar stories.

Hell, SOAP calls en general take longer time to get running than any JSON rest service call I’ve ever set up. Even when the XML interprets the same on both ends, though I’m not sure why that is exactly.

So it’s fair to say that I love JSON. The article outlines it as being easier to read, and there is certainly that, but for me it’s the efficiency.

Working with JSON never takes longer than it should.

> Working with JSON never takes longer than it should.

This is the key point right here.

JSON is a big reason I love groovy so much. No matter what json you have in groovy, you are a couple lines of code and a couple closures away from doing anything.

I always use json string payloads on HTTP aswell. It vastly simplifies rest service development.

Another thing I love about json is using it with cassandra. So easy to store big maps that hold the state of different things.

Dont even get me started on avro/kafka/etc....

You would have loved s-expressions in a typical lisp setup, then. :)

And you have also obviously never fallen victim to slightly off spec JSON parsers and folks that took advantage of them. Trailing commas? Definitely useful. Until they break a system. (Oddly, I have been stricken lately by how much people hate the extra parens of lisp, but nobody bats an eye at the pointless commas of every other language...)

That may be because you don't need to keep track of how many commas are there, you can just throw them in wherever you need one.

Yet parentheses must always come in pairs, you need a pair for every term in the program (brackets are only needed for full statements), and they tend to pile down at the end of complex functions and data structures.

I have seen more than my fair share of build failures from missing or extra commas. Worse, they have odd behavior in some languages.

That said, I cede that they are different. I just don't get the hate of parens.

My frontend builds fail because of trailing commas in the code (by design, so technically a non-issue). At the same time, my editors can balance parens but they can't really do anything but complain about trailing commas. It can't assume it's an issue and "fix" it because I just might not be done writing, yet. My editor can make more and safer assumptions about paren balancing, however, and so it does.

I really have no horse in this race, neither of these things bother me. I just can't help but notice that the paren problem in particular can be easily rectified.

Yeah, I concur. I mean, instances have a "global stream" where all posts from all users are sent? What's the use for that?

It would be like Facebook or Twitter having a "front page" showing all posts from all their users. I don't want to read what something posted merely because they're subscribed to the same physical server. There should be some classification of affinity by common interests.

> (Oddly, I have been stricken lately by how much people hate the extra parens of lisp, but nobody bats an eye at the pointless commas of every other language...)

I just checked an a comma is a grand total of three filled pixels on my coding font in my screen while a parenthese is 12 pixels. Parentheses are freakin visual bloat. Just look at this : https://i.imgur.com/UTGjbI5.png

I think that's a poor reason to choose a concrete syntax, and if it really bothers you it's easy enough to have your machine display them however you want.

For example, I use https://www.emacswiki.org/emacs/DimParentheses to make parentheses unobtrusive. I sometimes want to see them if they're at my cursor position, so I use https://www.emacswiki.org/emacs/ShowParenMode to highlight that one and its corresponding opener/closer. I use https://www.emacswiki.org/emacs/ParEdit to automatically keep parens/quotes/etc. balanced, move around whole subexpressions, etc. I'm sure there are similar tools for different editors (e.g. there's an incomplete list at https://github.com/shaunlebron/history-of-lisp-parens/blob/m... )

There's a screenshot of this setup at http://chriswarbo.net/blog/2017-08-29-s_expressions.html

> Parentheses are freakin visual bloat. Just look at this: https://i.imgur.com/UTGjbI5.png

Not to mention that f and 0.

We should write in Braille; f is just three dots and so is 0. Bonus: they are similar.

Note that parentheses do something, whereas commas are purely gratuitous. You can't just remove the inner parentheses in (1 2 (3 4) 5) without replacing them with something else. (1, 2, (3, 4), 5) is just stupid though.

Ah, Groovy. All the brevity of Java and all the type safety of JavaScript. I had no idea anyone still used it, let alone loved it!

That's an unfortunate stance on Groovy. Groovy code is often far more brief than Java's, and it has some beautifully expressive capability for making DSLs, both through its world-view on closures and its ability for the developer to walk and modify the AST at compile time (see, for example, the @Canonical attribute for how this can be useful)

While I, too, wish it built on a statically typed base, Groovy offers amenities such as @CompileStatic that go a long way towards alleviating this.

I personally find Kotlin strikes a wonderful spot with the brevity and expressiveness of Groovy, while managing to be more type-safe than Java (reified generics, null safety in the type system). But there is no denying Groovy's heavy influence on Kotlin, either.

Groovy may not be perfect but I don't think the picture you paint is accurate, either.

If you need to use the JVM ...

> walk and modify the AST at compile time

Clojure is best for this. A macro only a few lines long in Clojure will do the same as what a few hundred lines in Apache Groovy are required for. It's possible to walk the AST in Groovy but it's verbose and messy.

> CompileStatic that go a long way

Both Kotlin and Scala are both statically typed from the ground up, instead of having a @CompileStatic annotation tacked on later.

> the brevity and expressiveness of Groovy

Groovy started off in 2003 as a functionally similar clone of Beanshell, which had already existed since 1999, but with closures added. Someone then translated all the standard functional methods from Ruby into Groovy's codebase. Of course, Jython, a JVM version of Python, has been around since 1997. So Groovy is a relative late comer to the JVM in this regard.

Groovy tried to be a bit of everything for everyone on the JVM, but has ended up being used only for 20-line build scripts in Gradle.

I use Groovy with Spock to write unit tests. Wonderful way to write tests.

Pretty much wrong on all counts - doesn't sound like you know very much about groovy at all?

Have unfortunately had to use it as it was the scripting language embedded in an application we used. Now I avoid it whenever possible. Even BeanShell was better!

You’re comparing SOAP, a message protocol, with JSON, a data format. SOAP is so much more, it specifies how objects are defined, how they should be serialized and delivered to a server (WSDL). Note that JSON doesn’t have anything like this that has seriously taken off. There’s a few attempts, but they aren’t in great use or have tooling support. So as a result everyone working with JSON implements their own de-/serializer, resulting in all sorts of errors.

SOAP on the other hand takes care of this and even has a formalized date type and various number types.

Your complaint is about two vendors creating incompatible inplementations/extensions. I’ve run into this as well and causes a lot of issues with compliant implementations that cannot work together. I think this has been a big reason why eventually SOAP will die.

I think parent's point was that with JSON you are, in most langages, a JSON.parse(payload) away from a message protocol.

I am interfacing our startup's services with legacy (insurers) systems. We don't use the same programming langage. The docs are missing. Yet, we somehow manage to set up a functional API with minimal pain using JSON.

When I compare that, to the amount of documentation a SOAP service requires (for another partner), I find JSON much more usable

Yeah but most projects I worked on used soap as a data format. I had to interface with a french administration that parsed the xml manually (poeple printed and read it) and an american administration that used regex to read the xml.

Just having devs reading the dom correctly, including namespaces, is a rare occurence. It's sad, but we are living in a time were too many software is written, and not enough competent devs are available.

So simplicity, again, wins.

I did specifically say it wasn’t technically the languages fault, but you certainly have a point. I don’t think it’s completely unreasonable to bring in SOAP even though you’re talking about the language, because I’ve almost never worked with XML without also working with SOAP. But in not completely comparing it to JSON either, I mean, I love JSON for the efficiency of transferring it between systems, so I’m not completely comparing XML/SOAP to JSON the language either.

As far as lifecycle goes, it’s not actually the SOAP call but the way the two tech stacks parse the XML that’s the problem. You can also do rest calls with livecycle, and you’ll run into the same problems trying to pass XML created by any of the standard .net libraries to it.

I agree SOAP is terrible, but part of my point is that SOAP taints XML, because it’s a really common way to transfer XML.

I do use XML in other ways of course. We get quite a few datadumps in XML, that I turn to SQL through Microsoft SSIS, and even here it’s annoying. Not so much the language, but the way it breaks your SSIS service if it’s not delivered exactly as specified in its schema. Obviously that shouldn’t happen, but it does. Sometimes a supplier delivers half a file, and everything because SSIS didn’t get the XML specified in the schematic. (The fact that this breaks this is another wonderful story, where multiple datasets gets delivered within the same XML file.)

JSON and XML are languages, as you say, but their main usage (at least for me) is transferring data between systems and techs, and that is almost always easier, simpler and safer with JSON. Maybe it’s unfair to call JSON the better language because of that, but it’s certainly more useful to me.

> I mean, technically it’s just XML, but the two techs expect your XML elements to be build a certain way and break when they aren’t. So in order to make the two techs talk with each other through SOAP, we had to write a custom library to translate the .Net xml before we send it. And that’s just one in a long line of similar stories.

Something similar happens me with Java. Our product have some legacy ancient code that serialize/deserialize XML to beans using a library (JOX) that works with pure reflection (pre Java annotations).

I try to replace it for something more modern like Jackson or JAXB, but I hit a wall when I found that the XML that reads/writes these library are very different from what Jackson or JAXB expect to find. So, or we translate all XMLs to the new format or I write some translator to transform the old XMLs to something more standard.

It does. Its not that much different from Python dictionaries, but those are so much more pleasant to work with.

Trailing commas, comments (in large multiline dictionaries) and proper integers are some features I can think of. Small things like those can make a big difference in ease of use for many practical cases.

If we ignore syntactic aspects then the success of JSON is due to one fundamental reason. JSON relies on two structural elements [] and {}, which have very clear semantics of sets (collections) and tuples (combinations), respectively. It is simple, general, natural and consistent.

In comparison, XML also provides two structural elements but they do not have such a nice duality and clarity of interpretation. Here is a very typical example which is more than an anti-pattern:

    <books> 
        <book id="1"><title>XML is evil</title></book> 
        <book id="2"><title>XML considered harmful</title></book> 
    </books>

Here we see that semantically <books> is a collection while <book> is a tuple (where <title> is an attribute). Yet, they use one and the same structural construct. XML does not enforce the dual semantics of sets and tuples, and therefore XML files are frequently a conceptual mess. The above example is essentially wrong but XML does not care.

XML without schema are a mess.

XML with schema and validtion can be a joy to use compared to the untyped JSON world, but it's the lack of having to specify type which helps JSON remain popular.

Given a choice between strong or weak typing and weakly typed or un-typed systems will prevail in popularity.

Looking at javascript itself and the type coercion rules it is easy to dismiss it as unworkable mess which leads to expensive bugs in production, but in the real world the ability to mostly ignore types lets people get things done in a more hackish / amatuer way which eases the learning curve and helps popularity.

Typing (enforcing constraints) is an important aspect. But even without typing XML has one fundamental flaw. You are not able to (correctly) represent tuples with attributes which are sets. In XML, tuple attributes are properties, for example:

    <book id="1" title="XML"> -- object or object

Now let us assume that I want to have a list of authors as a property:

    <book id="1" title="XML" authors=["Me", "My Friend"]> -- NOT SUPPORTED

Therefore, we use a workaround (a crime actually):

    <book id="1" title="XML"> 
        <author>Me</author>
        <author>My friend</author>
    </book>

Now a book is a collection where authors are members (IS-IN relationship). This is not what we wanted. Our goal was to have an attribute "authors" which is a collection.

Pretty much any time I've seen something like this, relevant inner element of the book is:

  <authors>
    <author>...
    <author>...
  </authors>

Which reminds me of the picture[1] on the cat-v.org section about XML.

[1] http://harmful.cat-v.org/software/xml/xml_ascent.png

The article "XML Sucks" containing that Ward Cunningham image is interesting:

http://harmful.cat-v.org/software/xml/

It includes this classic James Clark quote:

“Any damn fool could produce a better data format than XML” – James Clark 2007-04-06

Which Tim Bray discussed in "Any Damn Fool":

https://www.tbray.org/ongoing/When/200x/2007/04/08/James-Cla...

Tim Bray recommended James Clark's Random Thoughts about "Do we need a new kind of schema language?": "If you’re doing original work around the intersection of messaging and programming, you need to read it and think about it."

http://blog.jclark.com/2007/04/do-we-need-new-kind-of-schema...

James Clark writes:

>Some people propose solving the XML-processing problem by adopting an XML-centric processing model, for which the leading technologies are XQuery and XSLT2. The fundamental problem here is the XQuery/XPath data model. I'm not criticizing the WGs' efforts: they've done about as good a job as could be done given the constraints they were working under. But there is no way it can overcome the constraint that a data model based around XML and XSD is just not very good data model for general-purpose computing. The structures of XML (attributes, elements and text) are those of SGML and these come from the world of markup. Considered as general purpose data structures, they suck pretty badly. There's a fundamental lack of composability. Why do we need both elements and attributes? Why can't attributes contain elements? Why is the type of thing that can occur as the content of an element not the same as the type of thing that can occur as a document? Why do we still have cruft like processing instructions and DTDs? XSD makes a (misguided in my view) attempt to add a OO/programming language veneer on top. But it can't solve the basic problems, and, in my view, this veneer ends up making things worse not better.

>I think there's some real progress being made in the programming language world. In particular I would single out Microsoft's LINQ work. My doubts on this are with its emphasis on static typing. While I think static typing is a invaluable within a single, controlled system, I think for a distributed system the costs in terms of tight coupling often outweigh the benefits. I believe this is less of the case if the typing is structural rather than nominal. But although LINQ (or at least newer versions of C#) have introduced some welcome structural typing features, nominal typing is still thoroughly dominant.

>In the Java world, there's been a depressing lack of innovation at the language level from Sun; outside of Sun, I would single out Scala from EPFL (which can run on a JVM). This adds some nice functional features which are smoothly integrated with Java-ish OO features. XML is fundamentally not OO: XML is all about separating data from processing, whereas OO is all about combining data and processing. Functional programming is a much better fit for XML: the problem is making it usable by the average programmer, for whom the functional programming mindset is very foreign.

More words from James Clark on JSON: "Yay" and "Sigh"

http://blog.jclark.com/2010/11/xml-vs-web_24.html

>If other formats start to supplant XML, and they support these goals better than XML, I will be happy rather than worried.

>From this perspective, my reaction to JSON is a combination of "Yay" and "Sigh".

>It's "Yay", because for important use cases JSON is dramatically better than XML. In particular, JSON shines as a programming language-independent representation of typical programming language data structures. This is an incredibly important use case and it would be hard to overstate how appallingly bad XML is for this. The fundamental problem is the mismatch between programming language data structures and the XML element/attribute data model of elements. This leaves the developer with three choices, all unappetising:

>live with an inconvenient element/attribute representation of the data;

>descend into XML Schema hell in the company of your favourite data binding tool;

>write reams of code to convert the XML into a convenient data structure.

>By contrast with JSON, especially with a dynamic programming language, you can get a reasonable in-memory representation just by calling a library function.

It is again a workaround because <authors> is a member of a collection - it is not a tuple attribute. Which suggests that we cannot enforce this separation and an application has to understand itself which element is an object property and which element is a member of a collection.

You're trying to impose an arbitrary model on XML.

XML grew from markup languages. It's as if we took some text, parsed it, and then store the resulting syntax tree along with the text. Here elements are non-terminals of the grammar and element attributes are additional augmenting properties of those non-terminals.

The text, of course, does not have to human-readable, it can very well be a sequence of anything, for example, of bits, bytes, words, etc. In XML we'd have to represent these with special elements, something like <byte value="00" /> or <int32 value="12345" />, but once we do this, we can use XML to enclose them into additional non-terminals that tell us what these bytes mean and let us use automated tools to manipulate them.

XML can represent objects, but in its own way: we have to first sort of serialize our object into a sequence and once we have this sequence, we can use XML. The model you seem to be talking about an abstract model of abstract objects in memory. Although memory is technically sequential, we normally ignore this and treat objects as nodes in some graph. This is not the domain of XML; it has to be a sequence to begin with. XML has a concept of IDs and references to IDs and thus can represent graphs reasonably well, but it must be a serialized graph.

So XML is basically a language to express the underlying grammatical structure of an arbitrary sequence. That structure is a tree, but it is based on a sequence nonetheless. It's not quite what abstract objects are in programming; but if you think of files, for example, files are sequences and thus are totally the domain of XML.

> It is again a workaround because <authors> is a member of a collection - it is not a tuple attribute.

Its an element not an attribute, but the set of elements and the set of attributes are both collections.

> Which suggests that we cannot enforce this separation and an application has to understand itself which element is an object property and which element is a member of a collection

It's true that that is not part of bare XML and the nature of attributes vs. elements, even though you want to impose it there.

OTOH to the extent that those words correspond to a well-rounded semantic distinction, it is arguably captured in schema languages, and not mere application-level knowledge.

Even more fun: XML attributes undergo "whitespace normalization"!

https://stackoverflow.com/questions/260436/preserving-attrib...

XML with schema (and namespaces, which schemata necessitate) is an immense pain to work with. XML with simple elements and no namespaces is actually fine to work with, as is something like thrift/protobuf. I don't think the problem is that XML has schema and validation, it's that it has bad schema and validation.

JSON does have Schemas and Validation (JSON-LD and RDF I believe are the relevant standards here), which are backwards compatible with parsers that don't support it (though the number of parsers that do is sadly rather low).

In principle you still get schema and validation.

Also https://swagger.io/

No. JSON-LD aims at aiding interpretation of JSON as RDF data (semantic web, description logic). It's terrible IMHO.

There is also JSON-schema.org if you don't like RDF/LD.

There is a draft of JSON schema: http://json-schema.org/. And there are working implementations.

Yes but in XML we view things as trees, not sets, tuples, or the likes. If you consider stuff as trees, there is not anti-pattern and the example is not wrong, it is correct. It is easy. Maybe we can be a little bit dissatisfied with the choice of attributes vs elements. Of course JSON and its small spec is easier than XML and its bunch of complex recommendations.

I think it's more appropriate to say XML is about a grammar description of some information serialization. Whereas JSON starts from concepts such as maps which are higher-level data structures. Also, SGML/XML originally isn't so much about trees as about regular tree languages - where the sequence of elements appearing below a node must satisfy a content model given by a regular expression.

Yes, you are right. So the question is which model is better: a tree model (with nodes as flat tuples) or JSON model with nodes being either collections or combinations (with arbitrary nesting). It seems that JSON is more general while XML requires you to map the reality into a tree representation. It is somewhat similar to the relationship between the hierarchical data model and the relational data model.

Rich Hickey has a good point in his Rails Conf 2012 "Simplicity Matters" talk[0] - json is just more simple than xml, as it clearly states which parts are maps and which are sequences. Xml has an inherent, implicit order in everything, due to its history as a document markup language.

If you think xml through the lens of jsx pragma style transformation, xml element and it's attributes are more or less a (typed) map. Similarly element's children is an (ordered) sequence. Where as in json you can have maps inside maps, in xml you always have to wrap maps inside an ordered sequence, as maps can only be passed to element as its children.

You can totally use xml without paying attention to the order, but the order is still there, complicating things.

[0] Rails Conf 2012 Keynote: Simplicity Matters by Rich Hickey - https://youtu.be/rI8tNMsozo0?t=28m5s

JSON maps well and intuitively to common datatypes: with a list and a hashmap, you can model a lot. Conversely, structs needing to be passed around in application code will probably be made up of stuff that maps well to JSON. XML's greater-than-blindingly-obvious impedance mismatch is a barrier to entry when an alternative, dead simple format exists.

The XML ecosystem nonetheless has parts that are desirable for certain use-cases, as the reinvention of schema validation for JSON, awkward namespacing, and efforts to dabble in hypermedia linking show. The XML ecosystem didn't require these enhancements and extensions to be used, but implementations in the wild often did, which was further off-putting to newcomers. This helped JSON gain ground back when it (and the ecosystem around it) was truly minimal and simple.

It also embodied the hacker, kludgey ethos and mythos, because without elaborate schemas and code generators, you'd pluck out only the fields you cared about and disregard the rest.

These factors led to JSON's rise at an opportune time, giving it credence and exposure, and within a few years every hip cool API was emitting JSON and using cool methods like PUT and DELETE and throwing around terms like REST despite serving linkless structs as 'application/json'.

The prevalence and staying power of CRUD HTTP APIs exchanging loosely typed JSON payloads shows that there's real benefits to semi-human-readable, self-documenting payloads that can be parsed into either rigid native datatypes, or loose collection types with ease. JSON fits into a world where stuff can be deliberately or inadvertently half-assed and still work.

This article had a footnote linking to a snide Douglas Crockford retort to a blog post that was dismissive of JSON. The other discussion comments on that page are a fascinating history of the arguments for and against, through the lens of 2006.

https://scripting.wordpress.com/2006/12/20/scripting-news-fo...

Whoa, thanks for linking that. It's really interesting seeing all this talk about sandbox escapes.

This is a particularly interesting quote:

> One huge practical advantage to JSON over XML in the web browser is that you can load JavaScript from any site, while you’re restricted in which sites you can load XML from. I have no idea why browsers insist on limiting where you can load non-executable XML from, when they don’t bother to limit where you can load executable JavaScript from, but there you go: that’s how it is, and there’s nothing anybody can do about it. JSON nicely works around that limitation, instead of holding its breath and waiting until the world comes around to being fair. Surely you can appreciate that practical advantage.

>Though one might think that a contest between data interchange formats would be unlikely to engender death threats, Winer wrote:

>"No doubt I can write a routine to parse [JSON], but look at how deep they went to re-invent, XML itself wasn’t good enough for them, for some reason (I’d love to hear the reason). Who did this travesty? Let’s find a tree and string them up. Now."

In retrospect, he should have said: "Let's find a tree and stringify it."

The success of JSON is due to one thing: Browser support. As soon as it's use on a server, or a mobile device, or practically anywhere else almost all the benefits of JSON go away.

Can you inspect the payload easily? Well, yeah, if you unwrap the bytes into a string or open up Wireshark. But on any moderately optimized system, the JSON is going to be gzip encoded and whitespace compressed, making in quite unpleasant to look through. By that point a pretty printer is needed, and you might as well have picked any format that has a pretty printer.

Can you add new fields easily? Yeah, but try changing a field name, or its type, or remove a field. This is guaranteed to break some client or server somewhere eventually. I've seen code that uses at least 5 permutations of "Zipcode", "zipcode", "zip_code", etc. I also wouldn't dare try to change it for fear of accidentally breaking our payments processing system.

JSON trades ease of starting up, for suffering when the code base gets big. It's an incredibly alluring trap, as TFA's graph shows.

Changing fieldName is not easy in any data format.

Schema has nothing to do with it. If your schema change, you are still going to to break existing client somewhere. It's not like schema can magically tell which field is renamed.

Schema also doesn't help different permutation of how a field is spelled. Because, in web world, nobody is going to use your schema. So you just end up with 5 schemas, each with different way to write "zipCode".

And if everyone can agree to use your schema, then they can agree to write "zipCode" the same way.

> Changing fieldName is not easy in any data format. > Schema has nothing to do with it. If your schema change, you are still going to to break existing client somewhere. It's not like schema can magically tell which field is renamed.

If you use something like thrift or protocol buffers then the field on the wire is numbered, and so you can change the human-friendly name that's used in code without breaking wire-level compatibility.

Why are you using wireshark to inspect json payloads?

Your other complaints stem from poor system design and lack of shared data contracts. They could be complaints about any data format.

What shared contracts are there for JSON? The only one I know of is OpenAPI/Swagger, and that's already complicated. Wireshark is used because sometimes we don't have the source code for software we use. The only way to debug is to see what's on the wire.

I would argue that the fact that json doesn't have schema support built in makes it more difficult to handle those issues though.

Avro json schemas and jackson json annotation for POJOS makes json object schemas easy to implement. Versioned object repos make great shared schemas and are easier to manage than xsds. (imo)

Anything would be better than XSD (though I don't think JVM-only is a good idea), but with an object you still don't necessarily know what kind of changes are going to be forward compatible or not. The only thing I've had any success with is something like Thrift or Protocol Buffers, where the canonical definition of your objects is a terse, Algol-like syntax but one that's been designed explicitly with version compatibility in mind, and you know exactly what changes do or don't break compatibility.

XML has browser support too.

So, when do we get a typescript for JSON? You mean a schema like XML has?

It looks like typesafe programming languages are on the rise, yet, when it comes to the format used for data exchange we go from typesafe to unstructured. I find that odd.

The counterpoint here is that a lot of the success of JSON was in presenting an alternative to the then-state-of-the-art, which was basically "here's a small mountain of spec documents explaining how you need at least fifty megabytes of metadata spread out over forty-three separate namespaces' worth of elements to safely send a single string key to someone else with XML".

And at the time people insisted JSON could never replace that, because JSON was so underspecified and unsafe. You could maybe get away with it for a couple requests to your puny baby child's "Fisher-Price My First Webservice" toy program, but JSON would instantly and completely fall over the instant any real-world problem presented itself.

Yet here we are all those years later, in a world which largely runs on JSON as a data interchange format.

Or, more concisely, as I wrote in 2006 in response to the Dave Winer piece someone else linked in another comment:

And now here are these kids with their startup companies and their weblogs who are getting data exchange and even things that kind of look like APIs out of… JavaScript arrays? The XML guys are sitting up on the mountaintop like the Grinch, with his pile of stolen presents, wondering how Christmas still managed to happen: it came without specs! It came without hype! It came without angle brackets, envelopes or types!

Yep. I'm reasonably new to software development, and I've always found JSON is simple yet effective. Less so for XML. It has its place, but I think less so for a simple and reliable data interchange format.

Or, "worse is better"

Worse is highly subjective: the XML world spent millions of dollars layering on huge amounts of complexity but never made a corresponding investment in having common tools not suck. Things like validation and schemas still have a horrible user experience and then you have things like libxml2 meaning that most of the world can’t use XSLT or XPath features added a decade ago. Similarly, it’s been notoriously hard to find things in the XML world which have quality documentation, examples, or working validators — doubly unfortunate given the complexity of what they expect users to do.

If you have a job to do, JSON is just plain better on almost every count. I remember being excited by XML 1.0 back in the 90s but the fall in popularity is something lots of people worked hard to accomplish by building so many new, more complex things rather than fixing the foundation. The assumption that users will have to learn your standard is almost always wrong.

Not really. There sure are some warts and inconsistencies in implementations (and across specs), but the success is because of the limited complexity. It's more "perfection is achieved not when there is nothing to add but when there is nothing left to take away" than "worse is better".

There’s passion for you.

Just like there is always a group of people turning every programming language into Java there are also people turning JSON into XML: http://json-schema.org http://www.jsoniq.org http://goessner.net/articles/jsont/ https://json-ld.org

You've got a point. I really like TypeScript but I haven't found a nice way yet to process the results of APIs that return JSON in a type safe manner. I'm surprised there isn't a standard way to do this yet.

GraphQL is similar to what you describe

Check out io-ts.

I use XSD a lot for XML validation in .net world and better intellisense when writing XML by hand. I was exploring the json world but I could not find a json schema technology that would be as expressive and the tooling around it is much inferior IME.

I feel like the technology around schemas (json, xml or anything else) is underrated and not developed enough. And if I wanted to support both json and xml it would be such a pain to maintain, would be nice to have a tool that could translate xml schema to json schema and back.

> would be nice to have a tool that could translate xml schema to json schema and back.

I once worked with a customer that bolted a JSON frontend onto an existing XML backend. However the translation definition was not predefined but determined at runtime. Some XML node could have 1+ children but because of this translation the result would either be an object (1), or an array of objects (2+). Needless to say the implementation in the consumer became rather messy, having to deal with this dynamic structure.

> would be nice to have a tool that could translate xml schema to json schema and back.

Hmm, after a quick search I found a bunch of json-to-xml and xml-to-json converters, including libraries. E.g. JSON-Java library [0]. May be same things exist for .Net. Is this what you meant?

[0] https://github.com/stleary/JSON-java

-ss

I think parent was talking about the document definition (schema) instead. So not to rely on the document instance for the translation definition.

Ah, got it. Thanks. I wasn't entirely sure, admittely not a very experienced json user. Does term "schema" apply to json in the same way it applies to XML?

/* Well... so... One can convert JSON to XML and apply XML schema verification. That's partly a joke :) */

-ss

I’ll pass on any serialization format that is itself vulnerable to RCE.

You can easily create a parser with RCE using just JSON. Just define a simple schema that looks like

  {
    "type": "ObjectName",
    …
  }

The only difference between this and e.g. YAML is YAML has specific syntax for declaring the object type whereas with JSON you need to declare it yourself.

I think the the only people who could be confused as to the reason for the rise of JSON are people who have never used XML

> By being at the intersection, [JSON] turns out to be the thing that everybody can agree on, and so it's really easy to pass data back and forth. Prior data interchange formats tended to try to be the union of all the languages, and that turns out to be horrendously complex and really difficult to deal with. JSON by being so simple actually became really easy to use.

- Douglas Crockford ('discoverer' of JSON)

https://youtu.be/-C-JoyNuQJs?t=730

What amazes me is that I regularly train people in IT that have never used JSON themself and are not sure what's it good for.

So something that is quite everywhere now, is simple, easy, and has existed for a long time, is still far from being a given.

It definitely puts in perspective people that are telling me that my webpack configuration from last year is obsolete, that I should ditch everything and learn Elm, that nix is the future of package management or that webassembly is going to change everything next week.

Of course, JSON has its own share of issues too, largely as a result of it trying to be as simple as possible. Many JSON parsers support nonstandard features such as comments and trailing commas, and in the process, create mutually incompatible "dialects". XML might be verbose and complicated, but I think JSON unfortunately is a bit too far in the other direction.

If a JSON parser ignores the JSON spec, is it the spec that went too far in the other direction? Or is that parser just badly-written? (Or misguidedly lenient?)

You could say the same for an XML parser that allows a naked '&' between tags, or continues parsing a document that omits a closing tag somewhere.

Your complaint is probably still valid, but I'd say it's better directed at the culture than the specs.

"JSON’s dominance is surprising when you consider that as recently as 2005 the web world was salivating over the potential of “Asynchronous JavaScript and XML” and not “Asynchronous JavaScript and JSON.”"

"As recently as 2005," heh. In the frontend world, you might as well say "as recently as the Paleolithic age."

It’s a great thing that JSON adequate.

It’s a great pity that JSON isn't much better.

It would be great if JSON had comments, more thorough typing and determinism so that data structures could be compared. Apart from that it’s streets ahead of XML.

It's exactly as good as it needs to be. Trying to 'improve' JSON will turn it into xml again.

Surely it's a problem that in JSON, duplicate keys are allowed?

https://stackoverflow.com/questions/21832701/does-json-synta...

Have you seen Zish? https://github.com/tlocke/zish It's designed to be an improvement on JSON, so it supports comments and has native timestamp, bytes and decimal types.

One aspect about JSON is the lack of comments. This makes it mostly unsuitable as a configuration language and in fact simple "JSON for humans" such as https://hjson.org/ and https://json5.org/ exist. In my opinion, this is a design flaw and if JSON would be a bit more human-oriented, it could be much more widespread in non-web contexts.

It is not design flaw, it was a conscious decision to enhance security. JSON is still the only secure (and fast) generic transport protocol, all others but msgpack are insecure by design. The various attempts to "enhance" json with types, datetime, comments, ... just make it worse, don't fall into that trap.

The latest JSON changes also made it more insecure, while they didn't clarify the missing spec points, so even they fell into that trap. msgpack is more also a bit insecure than JSON, but not as dramatic as XML, YAML, BSON, ...

I actually like that JSON has no comments.

With comments in a data transmission format you will end up with things like "conditional comments" in IE.

I agree there are certainly many better configuration languages.

I agree, but you can do context switchs also without having comments. For instance, the JSON Schema defines (Source: https://json-schema.org/latest/json-schema-core.html#rfc.sec...)

  {
     '$comment': "string that can hold whatever you want"
  }

and it is obvious that you can embed any other language in a string.

It's funny how people often compare JSON and XML, but don't mention one huge difference: XML is "more powerful" because it supports attributes on nodes, while JSON doesn't. So if you want to encode data structure with metadata on nodes (like objects with a type), you'll have to introduce a non-standard workaround.

Of course that's often not an issue, but I've run into it in the past. :)

http://www.flightlab.com/~joe/sgml/faq-not.txt

Q. I'm designing my first DTD. Should I use elements or attributes to store data?

A. Of course. What else would you use?

Q. What is the usual way to choose between using attributes and children?

[Update from David Carlisle:]

In England, the normal method is to flip a coin in the air and call heads it's attributes, tails it's children.

Unfortunately I'm not sure what to suggest in other countries where you can't guarantee having the Queen's head on one side of every coin.

Hilarious, but the deeper rule is: when there isn't a concept of rendering a document to the user (where attributes hold rendering properties not directly presented to a user) the concept of a markup language doesn't make sense in the first place. So if you find yourself in a situation where you need to decide on encoding data as elements vs attributes, you probably shouldn't use SGML/XML at all.

But then there is the wealth of XML standard data representations for financial/fiscal/e-commerce, medical, etc. data. You're deciding to use them to have your interfaces being implemented once and for all, and be able to switch eg. payment providers by merely specifying new endpoints and credentials. JSON doesn't really compare to XML/XSD in this space.

It's funny how people often compare JSON and XML, but fail to see that XML/SGML is for document data (like whole web pages) whereas JSON is a clever reuse of JavaScript object literal syntax.

Oddly, attributes do little to actually help you here. What mattered was that xml gave you a language that could describe a valid document. Which, for reasons that actually exist, wasn't necessarily a valid xml document.

That said, the effort to write a valid schema was rather high. And it was sold with the other technologies that were in violation of the anti-fragile spirit of the web. That left us with people wanting something different. JSON was born as much for basically being javascript literals minus functions as it was anything else. Most any other explanation is just someone selling you something.

(Now, attributes did offer a technical way to ignore subsets of the attributes that would be hard to do any other way. But they didn't exactly make it easy, and namespaces did not compose nearly as well as people hoped they would.)

JSON was just a quick shot by Douglas Crockford. No need to spin a genesis myth around it.

I don't know what genesis myth you mean. I do know that I was regularly serializing JavaScript literals for years before I heard of json. Basically just said I could no longer serialize functions. Or references. Which, to be fair, was not that common and I could easily live without.

I do miss being able to have comments.

XHTML is still alive as the section 13 "The XML syntax" in the "HTML Living Standard" shows us. Moreover XML comes with a host of useful technologies : schemas, XSLT, XQuery and namespaces. It is quite complex compared to JSON but more powerful when you consider the whole ecosystem. In the end, it depends on the need we have. Quick, small, simple XML is possible and easy.

I feel like the best takeaway here is that XML is a document format, and JSON is for programmatic interchange, e.g. APIs. That's at least how I operate, and I use both XML and JSON for various things. My APIs are all JSON, but if I'm serialising some data to disk, or for export etc, it's usually XML.

JSON is one of the best things to ever come out of the CS disciplines.

It is one of the most perfect technologies I deal with.

Well, unless you need to deal with datetimes or timestamps. And it suffers from the same repeated-metadata problem that XML does. It’s easy for JavaScript users, that’s it’s main purpose. For serialised data either at rest or on the wire there are dozens of better formats e.g. HDF5. Or even ASN.1.

> It’s easy for JavaScript users, that’s it’s main purpose.

I'm writing Go applications that talk to Python applications with (you guessed it) JSON-over-HTTP. Saying that JSON is only for JS users is like saying the iPhone is only for people that need to make calls.

> For serialised data either at rest or on the wire there are dozens of better formats e.g. HDF5. Or even ASN.1.

A huge advantage of JSON is that it is human-readable and human-writable without extra tooling (although an indenter sure makes things easier on the eye). Also, I recall ASN.1 parsers being a rich well of RCE bugs.

Repeated metadata is just one of XML serializations, although the most common one. But there are technologies to compress XML and compression based on schema may give very compact results.

I too admit liking JSON for it's simple yet extremely useful in a whole lot of usecases, but calling it near perfect is a maybe bridge too far. Officially it doesn't have a representation for nan or infinity, otherwise perfectly valid floating point numbers.

Not: as groundbreaking as Scribe (Brian Reid at CMU) was, I don’t think SGML was inspired by it. They had parallel dev paths...

A message format that requires parser to understand both UTF8 and UTF16, cool!

Unsurprisingly, one of the most common data types is text strings, so most serializations need that anyways.

> only one API exposes data in XML rather than JSON

They are not mutually exclusive. It should be trivial to offer the data as either JSON or XML at the user's request.

Reply on news.ycombinator.com