JSON vs. XML with Douglas Crockford
corecursive.com
corecursive.com
[0]: https://news.ycombinator.com/item?id=35467711 [1]: https://news.ycombinator.com/item?id=35452861 [2]: https://news.ycombinator.com/item?id=35429976 [3]: https://news.ycombinator.com/item?id=35427394
Here's my hypothesis: they initially implemented it the obvious way on mousemove (or whatever similar immediate event). Then they a) noticed the horrible zipper noise that came from doing that, b) felt dread at the idea of plumbing into the webaudio API, c) said "aw fuck it" out loud and d) bound the event to mouseup.
Does the dev of that framework post here? If so, am I close on this one? If so I can commiserate from the opposite end-- I spent a long time getting audio stuff like this just right and then fudging text alignment for lack of juice.
The reasons are very interesting: your brain does a Fourier Transform (sort of) in real time to listen to audio. It allows you to separate two people or two instruments producing sounds at the same time, and abrupt changes will cause this transform to product artefacts.
The right way of doing it is to always make progressive changes to the gain level. This property is often used in audio engineering to "add or remove punch" in recordings: you can decrease the attack level to make the gain changes more aggressive and add punch, or lower it to make it smoother, over do it and it sounds muffled. In more concrete words: use the setTargetAtTime WebAudio API.
For example, you can explain to someone with not much signal knowledge why a 15kHz square wave sounds exactly like a 15kHz sine wave to humans by looking at the harmonics through a Fourier Transform.
I do believe most of the magic happens in the brain though, the cochlea is a super advanced biological microphone, but the brain is responsible for harmonic and rhythmic processing. The brains figures out what is a voice, an instrument, or a percussion. My understanding is that it stores short-term temporal information needed to make sense of realtime information: a note alone might be perceived differently depending on the harmonic and rhythmic context ("wrong note").
I don't think this is true at all. The reason that JSON won is stated quite clearly in the interview: you do not need a software stack to handle JSON. This feature sells itself for any software development team who needs to have frontends running in a browser consuming data from a backend. You do not need an evangelist to convince people to use a feature that is ubiquitous, free, extensively tested, and more performance than anything in the market.
Particularly in the early web, the need for async "liveness" of pages didn't exist, so pages didn't consume backends directly. Everything was handled on refresh or navigation and server rendered as HTML. It was only after AJAX-style web apps took off (post-Gmail, really), that JSON started to get real traction. There were no JSON parsers in C#, Java, Python, etc initially. XML was far more common and SOAP was generally used for cross-endpoint communication.
"But you HAVE TO have a virtual DOM. You HAVE TO write all this boilerplate and ship a fat runtime. You HAVE TO manage the caching details to these performance problems and abstraction leaks (looking at you, useMemo!). You have to WRAP AND HIDE the HTML and CSS—which breaks accessibility, so you need to learn all the implementation details about a11y and manually add it back one ARIA role at a time (or just assert it's too hard, so less accommodation)."
All this when there are alternatives that are much simpler, so much easier, so much less code, and so much more consistent, folks constantly downplay it. "It CAN'T be used for large projects. You HAVE TO make this large object hierarchy to make web apps. You CAN'T rely on a compiler, because that's magic and no longer 'pure' JavaScript."
I especially love the "magic" argument since it completely glosses over the fact that we're providing info to processed sand we hit with lightning to make it think. Magic is already part of the bargain from your first "Hello World."
Only now it's not "is that a standard?" It's "everyone's doing it the unnecessarily hard way, and we've invested too much time learning all the ins and outs of the unnecessarily hard way, so if you want a job, you'd better get used to the hard way and suffer as we have."
> I think Object-Oriented Programming was an important advance over procedural programming, which is what we had been doing, but we’re stuck. We’re supposed to change paradigms every generation, and we haven’t. So we’re way behind now. Object-Oriented Programming hit the mainstream in 1980. We’re in the 2020s and we’re still there. We should have moved on.
1. In order to avoid ambiguities they'd need to use concrete vocabulary.
2. Any non-trivial program will need to consist of multiple input statements. Those input statements will have some kind of relationship to each other. This is roughly what a grammar is about.
3. Those future programmers will somehow need to check if the program does what it's supposed to do. They'd run it with test data.
Combine 1-3 and see that we just described, what a programming language and its ecosystem (compiler, runtime, etc.) do. Thus, programming languages are nowhere near to die out, they will just shift things higher up the layer lasagne.
We should move on when we find something that is objectively better, not because some years have passed.
And honestly, the only new thing that seems to be better than the current paradigm is Rust borrow model, which is not against object orientation, it compliments it.
The 90s-flavor OOP we're stuck with in many places is objectively bad for performance (see e.g. Mike Acton's takes on what he calls "Data-Oriented Design") as well as readability/understandability (where I'd say languages with a culture around more FP concepts and immutability have a way better track record) and concurrency (see Erlang/Elixir).
And, as always, you can simulate the useful properties of other paradigms, but that only goes so far. I think what the original quote laments is that the median software developer is leaning way more on "we've always done it this way" as a way of guidance instead of better ideas replacing the old ones.
Edit: Also: I totally agree with you about Rust, but I can't help but notice that Rust mainly stops you from doing dumb things (no inheritance, borrow checker…), which effectively dials back many of the… more unfortunate concepts/idioms of 90s OOP.
We truly live in a world
The API was a post with a single JSON field called data, with a CSV string value. You just know they were asked by management to deliver an API and decided to do quite literally the minimum viable.
They had 'no plans' to expand it.
So you get the XML header with UTF8 encoding, a root tag, an indented child tag. 3 lines. Then ye olde CSV, with the 5 required entities for ascii escaped, but if you manage to put accented chars in it, they end up in ISO 8859-1. Then 2 lines footer.
If you wrote out the file, the header and footer had to be byte for byte identical. Entities encoding non ascii just crashed the parser. Of course, the CSV lines were sensitive to indentation, they just dropped the first 8 chars of each line.
Still, XML was officially supported.
But... What do would you do instead?
Yet I get so many questions like "but we base64-encode them first?" or similar. One dev mentioned he'd never done such a "raw" POST.
Not surprisingly, they didn't put any example JSON in their API documentation.
I'll probably listen to this at some point and genuinely like hearing about the origins of JSON from Crockford himself.
Apart from that, I'd be surprised if this adds anything to what I've already heard and read ad nauseam, including from Crockford himself.
JSON is just a nice, simple serialization format that compresses well and interoperates acceptly with most programming languages. Most importantly with the language that it was designed to be consumed by: Javascript.
If one needs better performance or better guarantees about serialized data types, I guess one should use Protobuf, and/or ASN.1 (was that even correctly spelled?) or any of the countless other formats I have no clue about.
If you need better schema restrictions and interconnected entities + circular references: use a database, I guess.
At least everything XML has done in this space has more or less failed (e.g. SOAP). Well, failed might be too strong a word.
But all the work on defining data types and schema seems almost independent of the underlying syntax.
Since I've now spent too much time on this uninformed first comment, I will now listen what Douglas Crockford wants to say. But I think I've already read this story in another form a couple of years ago.
JSON is really a good example of "worse is better". E.g. an OpenAPI spec with automated client code and TypeScript definitions is not much different from SOAP on the surface level.
1: text with markup. Does not need all the XML machinery, but tags make a lot of sense here.
1b: Arbitrarily nested stuff
2: extensability by third parties without coordination. Namespaces as a central concept help out a lot here, and this use case can even justify some of the more insane features of XML (downloading remote schemas etc)
If you don't need either of those things, you won't get much value out of XML. If you do need them, XML can be neat. Certainly easier than trying to allow people to add arbitrary data (with validation) to your protobuf. It's just that these use cases are somewhat rare
In 2023, Python lxml can't handle multiple documents per file. And no one has any suggestions on how to handle this other then "break on the new document tag". Other libraries didn't look promising in other languages.
Then if you try to read it, you need to disable all the DTD stuff because it immediately tries to open other local files on disk...which it can't find. There's some corruption in how it parses DTDs because I had to implement a custom DTD handler in my jupyter-notebook to read it.
I could blame the dominant XML processing library of Python, I could blame the lack of attention XML gets, but really? This is just another contribution in a long line of XML not working in some way in every language as soon as you're not using that languages specific XML flavor.
Java and JSON is a different story. There was a window when the only JSON parser Java shipped with was JSON.parse() in its Javascript interpreter.
https://www.reddit.com/r/programming/comments/eaqgk/java_is_...
"I Wanna Be <![CDATA[" Sung to the tune of “I Wanna Be Sedated”, with apologies to The Ramones.
I can't seem to find it, but there's a serialization that's basically Python, so
Stuff(a=10, b={"foo": bar})
Unless I misunderstood you, you can use tagged unions[1] for this so not sure JSON needs any change there. Slightly more verbose but shouldn't be an issue in practice.
ProtoBuf is a horrible format. its not self describing like JSON, when it easily could be, without costing any extra bytes.
like if I have a message inside of a message vs just a bytes field that happens to have a valid protobuf?
Bwhahahaha good one
SVG
DOCX
SAML
JSX
The legacy of XML incalculable and expanding.
Xml excels when the content must be human consumed and machine readable.
I enjoy JSX as well (and even SOAP!). That's why I weakened the quoted sentence in my comment.
When I hear "JSON vs XML" I can't help but mainly think of data payloads in web applications, especially non-static client/server communication.
I wouldn't want JSON instead of SVG either :)
>Adam: That was it. That was the creation of JSON, which everyone is using today. But back then, everyone rejected it.
>In some ways it was a marketing problem. On one side, you had Doug, trying to convince customers that they can build interactive applications on the web using JavaScript and this simple thing called JSON. But on the other side, you had XML that had these big companies behind it, IBM, Microsoft, and big consultants. And later they even had some tech influencers like Dave Winer.
>Douglas: He’s someone who should have known better. He had a website called scripting.com. His style of scripting came from a clever program that he had written for the Macintosh called Frontier, in which he had a scripting language and an outliner and a word processor and a database, all in one program. And the idea was that you could do virtually anything in Frontier with a little bit of scripting. And he was also one of the big promoters of SOAP, the Simple Object Annoying Protocol.
>I don’t remember what the A was, but it might have been atrocious or abominable, I don’t know.
>But SOAP was a big deal at the time. They were right in wanting simplicity. They didn’t accomplish it, but they put simplicity in the name as sort of an aspirational thing. And so, when I started showing how JSON works, he was really threatened by that. And on his website, which was well-read at the time, he complained that, “this isn’t even XML. We should find who did this in string them up now”, which was a really ugly thing to say.
>Fortunately, nobody listens to Dave Winer, so I’m still here.
Dave's done some brilliant influential stuff, which Doug credits and I've written about before, but being annoying is Dave's brand, so Doug's "SOAP" joke is dead on. It's just as fair as referring to Marc Canter's "People Aggregator" as "People Aggravator".
https://news.ycombinator.com/item?id=20780928
https://news.ycombinator.com/item?id=21170440
https://news.ycombinator.com/item?id=16224154
https://www.gawker.com/183778/people-aggravator-marc-canter-...
Thanks a lot for the money quotes! Casual conversation in English is still sometimes hard to listen to for me without focusing a lot.
Fascinating concept. I found a bit more info in an article about UserLand Software, which Dave Winer founded after leaving Symantec.
https://en.wikipedia.org/wiki/UserLand_Software#Frontier
> In January 1992 UserLand released version 1.0 of Frontier, a scripting environment for the Macintosh which included an object database and a scripting language named UserTalk. At the time of its original release, Frontier was the only system-level scripting environment for the Macintosh, but Apple was working on its own scripting language, AppleScript, and started bundling it with the MacOS 7 system software. As a consequence, most Macintosh scripting work came to be done in the less powerful, but free, scripting language provided by Apple.
> UserLand responded to Applescript by re-positioning Frontier as a Web development environment, distributing the software free of charge with the "Aretha" release of May 1995. In late 1996, Frontier 4.1 had become "an integrated development environment that lends itself to the creation and maintenance of Web sites and management of Web pages sans much busywork," and by the time Frontier 4.2 was released in January 1997, the software was firmly established in the realms of website management and CGI scripting, allowing users to "taste the power of large-scale database publishing with free software."
> Frontier's NewsPage suite came to play a pivotal role in the emergence of blogging through its adoption by Jorn Barger, Chris Gulker, and others in the 1997–98 period.
> UserLand launched a Windows version of Frontier 5.0 in January 1998 and began charging for licenses again with the 5.1 release of June 1998.
> Frontier subsequently became the kernel for two of UserLand's products, Manila and Radio UserLand, as well as Dave Winer's OPML Editor, all of which support the UserTalk scripting language.
example: instead of the following:
[{"field1":"value", "field2":"value", "field3":"value"}, {"field1":"value1", "field2":"value2", "field3":"value3"}, {"field1":"value3", "field2":"value3", "field3":"value4"}]
If you could have this instead: [{"columns": {"field1", "field2", "field3"}, "rows": [{"value1,"value2","value3"},{"value1,"value2","value3"},{"value1,"value2","value3"}]
If json parser could then have the smarts to return the values by position for the various field names even though each row does not include the field names.
The only thing holding you back from doing that is 1kb of JS as the second argument to JSON.parse
But XML is not necessarily that bad for many things.
I still use this XML editor I made in 2008: http://move.rupy.se/file/logic.html
However they seem to be against it primarily because it's not JSON (keys are explicitly unordered) and that means some generators might be unable to produce valid output, so interoperability is tricky.
[1]: https://github.com/json-schema-org/json-schema-vocabularies/...
>Douglas: Some of the people who were doing JavaScript, were doing it badly and didn’t want to be told that they were doing it badly. There is this notion among second rate programmers that the most important thing they do is express themselves. It’s not making programs that work well and are free of error. That’s way down on the list. What’s much more important is expressing themselves, that, I’m an artist and I express my arts by leaving out semicolons. It’s that kind of thing. So what I was saying did not resonate with that at all. So that audience was not happy to hear it, and JavaScript was just not being used.
But its a painful slow process as there are not enough of them practical innovators. Big corp and consultants have an amazing ability to keep things stagnant for ages and milk complexity.
But the json revolution is definitely unfinished. Standards are important and so many domains have adopted XML already. A practical way to jsonify all these specs would be really useful.
genius? lol
he is certainly not humble and he's no smarter than anyone else. right place, right time.
his LACK of humility is what gave him the persistence required.
At no point did I perceive him to assert any sort of personal superiority.
While I use his JSON creation heavily, JavaScript as such is the second least attractive language, after PHP, in my view.
this podcast is the most humble I have ever heard him, to be fair, but make no mistake; DC is not a humble person.
A humble person would not have said "JSON was my/our idea" or "I was right"; a humble person would not be on a podcast talking about themselves at all.
JSON is not foundational, actually harmful because it ditches the main idea of web (linking things) for pushing around schemaless hodgepodge data structures...
Linking and schemas and validation and all that stuff are terribly important but a good and simple approach would ADD them when needed, not inflict you with maximum complexity for even the simplest thing.
In this respect json-ld has shown one way this can be done but overall its a problem that still asks for a brilliant innovator to solve.
There is no complexity in linking, there's nothing simpler than RDF. The whole reason you think it's complex is... drumroll... JSON. It made these bespoke unlinkable structures proliferate and de facto standard. Later JSON-LD tried to bring it back to sanity but because it still has to be JSON (because devs now hate everything else) the end result is complex, inscrutable (contexts?? @reverse???) and usually misunderstood.
A good design would allow progressively adding semantic complexity iff needed. I think the popularity of json was just because xml violated this very simple design aspect.
Unfortunately it is true that linking, metadata, ontologies and all that stuff are required as the ambition of information exchange increases. People who deny that obviously have never worked on any domain that requires significant data exchanges between many independent actors.
So now we are stuck with two sub-optimal solutions: a simple but incomplete json approach that must reinvent the wheel without destroying its good bits and a complex and hard to use xml approach that is only loved by software bureaucrats...
Ah, but we have schemas now! https://json-schema.org/
I've been able to convert our XSD's 1:1 to JSON Schemas, including choice elements. I'm sure there's not a 100% overlap but most XSDs could I think.
When the hot new thing in web dev is...
...
...
JavaScript XML (JSX)
> https://facebook.github.io/jsx/
So yes, it's not XML.
It's JavaScript XML.
This line made me think about the many discussions about Python syntax and the value of having brackets to know when a code block ends and starts.
Balanced named tags are another step to make something easier to visually parse, at the cost of having much more text.
Since when did Yahoo lose its "!"?
In reality however, often the best way to understand a problem in those configuration monsters (or the only way...) was to recompile the non-xml part with some tailored ad-hoc diagnostics. But the idea that the configuration was supposed to change a lot while the code is immutable proved surprisingly robust against ongoing demonstrations of exactly the opposite happening.