In Defense of XML | Hacker News Reader

In Defense of XML

blog.frankel.ch

136 pointstannhaeuser254 comments

I'm a bit annoyed that the author of the article seems to be imply that the move away from XML is just due to hype.

Also the point about the ad-hoc client validation doesn't seem right to me. If the server changes the XML response, the client may be able to fetch the updated schema (assuming the devs didn't forget to publish the update...), but then what? You still have to handle the new or updated fields in code somehow.

My biggest gripes with XML are twofold:

- It's as annoyingly verbose as Java is; I just get lost looking at all the tags (the namespaces don't help).

- The community seems to have this obsession with solving every problem under the sun with more XML and more specifications. I believe, this is wrong. Trying to standardise across the whole range of data interchange and configuration needs is the wrong approach, IMHO. JSON wins here, because it gets out of the way. I do wish JSON had comments and that JSON schema was better (for when you really need it), but nobody ever had an expectation that keys are unique across all JSON documents in existence, something that XML somehow insists is necessary for its elements and attributes.

But what the author is right about is that YAML is an absolute nightmare, and I can't understand why it continues to be so popular. Apart from the dozens of ways of writing booleans or strings, it's also way, way too easy to forget what level of indentation you're currently at, especially if you're navigating large files like Kubernetes or CI configs.

I think more projects (at least larger ones) should adopt their own DSL for configuration. For example, I find the terraform config format much cleaner than any of the YAML stuff.

The biggest advantage of JSON and YAML is that it has a very straight-forward object model consisting of literals, lists and dicts. All of these constructs map very easily to native data structures in most languages: you can just "load" and "save" an arbitrary json or yaml document.

The XML DOM is very different and does not map in an obvious way to objects in any programming language: you either use a DomDocument or object serialization for a subset of XML.

This alien object model makes programming with XML painful and verbose for no gain.

Sure: standards for xml schema and xpath are pretty nice but there are json equivalents which don't have to deal with elements and attributes. If standardization around YAML and XML was equally good would anyone pick XML?

> way too easy to forget what level of indentation you're currently at

This is easily solvable by tools. How is it worse than getting lost in braces or open/close tags?

> I think more projects (at least larger ones) should adopt their own DSL for configuration.

Please don't. JSON, YAML and XML config files can be easily and safely manipulated in all programming languages, for custom DSLs people end up using brittle regexps. It's not much better than putting config consts in a code file.

> This is easily solvable by tools.

Maybe, but I want it to work with standard tools. Most editors seem to arbitrarily jump back a couple of indentation levels e.g. if you add newlines somewhere. This obviously doesn't happen with JSON, since the editor won't have to guess whether you wanted to de-dent or not, it's all explicit. And I'm not making this problem up, it's literally a problem almost every time I have to edit some larger YAML file.

I'm not a fan of whitespace sensitivity in programming languages either, but at least Python etc. make a habit of discouraging 15 layers of indentation.

> Please don't. JSON, YAML and XML config files can be easily and safely manipulated in all programming languages, for custom DSLs people end up using brittle regexps. It's not much better than putting config consts in a code file.

What? Why would you parse a DSL with regex? I'm not saying people should write crappy DSLs. You can either write your own proper parser or you just use an embedded DSL, since there are multiple languages out there who are quite good at providing DSL support. The bonus is that it's often even type-safe, without having to be as mind-numbingly obtuse as XML.

On the note of type safety I can always suggest people to take a look at Dhall.

https://dhall-lang.org

Yes, this is something I've wanted to check out for a while now, I just don't think I could convince other people on any team to look at it. :D

I've had the same experience. Different editors and IDEs seem to handle YAML in wildly different ways. If you copy/paste a YAML stanza into a different document there is no telling how the editor is going to indent it. And trying to fix it can be something of a nightmare, especially when the editor indents different lines in the pasted text differently.

> What? Why would you parse a DSL with regex?

Let's say I want to process a nginx or apache config file programmatically to change a flag. Am I going to parse the full syntax? No, I'm going to hack something together with regexps.

> Sure: standards for xml schema and xpath are pretty nice but there are json equivalents which don't have to deal with elements and attributes.

Not quite there yet. Jsonpath is still not standardized (although a colleague of mine is working on it) and the many implementations out there disagree with each other.

(That said, I do agree with the general sentiment against XML)

There's DomDocument for json: https://docs.microsoft.com/en-us/dotnet/api/system.text.json...

For yaml: https://github.com/jbeder/yaml-cpp/blob/master/docs/Tutorial...

"The biggest advantage of JSON and YAML" should be put in a context of, "when working as a programmer with config files".

When we designed XML, we were working on things like a million pages of Unix documentation, or the flight manuals for aircraft, or research journal articles, or transcriptions of inscriptions found on fragments of ancient Greek pottery - lots of small similar documents, or very complex documents.

XML lets people who are not primarily programmers do sophisticated text processing and information design without depending on outside resources.

People who complain about XML being brittle are saying they want a system that forgives syntax errors, in the same way that JavaScript... doesn't... or that JAVA... doesn't... or that C... doesn't...or that Python... doesn't.

XML is unbeaten for the things it's good at. It was (still is, often) a problem that people used if in places where it wasn't so good. But that doesn't make it bad - it means you need to understand where it's good and where it isn't.

As the old saying goes - Use the right tool for the right the job!

To me, one of the virtues of XML is that, if the server changes the XML response, you can capture it, and a human can eyeball parse it and figure out what went wrong, and figure out how to change the code on the receiving end to compensate. It's hard to automate, sure, but it's far better than having the server hand you some updated binary format.

Annoyingly verbose: Yup. Absolutely. I thought XML was kind of OK. Then I learned some about S expressions, and looked at XML with those eyes, and it was absolutely disgustingly verbose.

> To me, one of the virtues of XML is that, if the server changes the XML response, you can capture it, and a human can eyeball parse it and figure out what went wrong, and figure out how to change the code on the receiving end to compensate. It's hard to automate, sure, but it's far better than having the server hand you some updated binary format.

Maybe I'm misunderstanding, but it seems to me you can do the same thing with JSON.

Sure. It's just that, during the days that I had to do much with XML, JSON wasn't really a thing yet. I'm comparing XML to the binary formats that came before it (in wide use), not to JSON that came after it.

My biggest gripe with XML has more to do with how data is structured: XML offers both tag attributes and child tags and when encoding data it's not obvious which one to use. You can also use multiple child tags and it's not obvious whether you should mix multiple kinds or not.

The usage is obvious when using XML as markup, but when used for data it's left to the one creating the particular spec.

You can have style guidelines, but other data formats don't really need that to answer those questions. More flexible is not always better, especially in languages.

>> XML offers both tag attributes and child tags and when encoding data it's not obvious which one to use.

>> The usage is obvious when using XML as markup, but when used for data it's left to the one creating the particular spec.

Either approach is valid. Having a choice about how to structure the data is not a bad thing.

My preferred approach is to use attributes for simple key-value pairs that are mainly used to select structured data found in child tags. If you have configuration data that is stored as XML, you could use attibutes to mark where the contained structured data should be used. For example: 1) language localization data could be stored stored in XML with attributes describing the language or locale; 2) build configuration settings might be stored in XML with attributes describing the target platform or CPU architecture.

For more formalized data formats, create an XML Schema that defines what the XML data should look like and then you can easily validate that the XML data is structured correctly and even generate parsers or XML to programming language bindings.

> Either approach is valid. Having a choice about how to structure the data is not a bad thing.

I though the point of using a well defined markup language was exactly that it imposed a clear structure on your data.

In another sense, xml gives no options: your data is going to live in a string.

>> I though the point of using a well defined markup language was exactly that it imposed a clear structure on your data.

A clear structure can be defined with an XML Schema Definition (XSD) document:

https://en.wikipedia.org/wiki/XML_Schema_(W3C)

An XML Schema Definition defines the structure and allowable content for a given XML document and allows XML data to be programmatically validated to ensure it adheres to the specified format and content.

>> In another sense, xml gives no options: your data is going to live in a string.

This is true. XML is inherently text-based. There are options for binary data formats depending on your needs.

"it's not obvious which one to use"

Perhaps this explains one system I saw that included entire encoded XML documents as attributes in other documents. On the plus side this embedding only went one level deep.

Now I'm curious. Is that Microsoft Office with embedded documents (e.g. a powerpoint presentation with a word document embedded)? I've never looked at how the pptx format contains the docx format internally.

I've seen it in XML-based docs showing example XML responses. Then somebody tried to document the doc-generating-tool and I think we all know how that went.

Child elements are different from attributes because elements can be structured (with attributes, other elements etc.) and repeated within the same parent element. On the down side, elements are a little more verbose and whitespace needs some attention.

This restricts attributes to zero or one unstructured values, which is more than a "style guideline".

It is questionable even in markup. Attribute restrictions (unique, can't be structured) gave rise to minilanguages

    class="foo bar"
    lang="en-US"
    style="margin:0;padding:0"

XSLT blows away promise of structured data with XPath and functions like concat in attributes. And it is a special case in DOM and XSLT making it much harder to manipulate.

Meanwhile <script> and <style> content hidden by default.

> Meanwhile <script> and <style> content hidden by default.

This is a fun thing that most people don’t realise. You can totally do this in CSS:

  head, title, script, style {
    display: block;
  }

And bam, your document title and the contents of any inline scripts and stylesheets are now visible on the page.

There's no ambiguity at all. Attributes are what fields are to a record; basically everything goes into attributes. Child elements are other records and are used when either there's more than one or their order is important or they also appear as children of other elements. Text is only used when there is actual content for humans that XML delineates; when XML is used as "application/xml" :) data there should be no text (unless parts of your data are such text).

More than "fields to a record" the difference between attributes and elements is that attributes cannot be nested. So in a sense if you are trying to write JSON in XML all base values should be in attributes.

>- It's as annoyingly verbose as Java is; I just get lost looking at all the tags (the namespaces don't help)

I guess if you take the effort to make it as annoyingly verbose it is, but in regards to namespaces for example Joe English's email on the psychopathologies of namespace usage http://www.flightlab.com/~joe/sgml/sanity.txt if followed for best practices also has the side effect of reducing much potential verbosity.

> Also the point about the ad-hoc client validation doesn't seem right to me. If the server changes the XML response, the client may be able to fetch the updated schema (assuming the devs didn't forget to publish the update...), but then what? You still have to handle the new or updated fields in code somehow.

To be fair, there are often tools that will generate object definitions in the target language from an XML schema. It's not a perfect solution - you need to rebuild the project when upstream changes - but it's not like this hasn't been addressed. For what it's worth, Protobuf has the same property.

Yes. And then you end up with a domain-model that is badly designed because XML is not capable of expressing rich and nice-to-use data structures (as is possible both in OOP and in FP languages). You end up with lowest-common-denominator stuff, at which point you might just as well use JSON and parse it into something type-safe yourself (or not, and just use it as a [String: Any] dict).

Matrix protocol uses JSON with namespaces, both Java-style, C and XML-style namespaces.

And because it's non-standard it's irritating. Every JSON parser doesn't have this built in, so you have to implement it at the application layer if you want it.

> - It's as annoyingly verbose as Java is

I reviewed an XML-based international industry 'standard' that allowed both verbose and terse versions of the same tags. Example files I saw tended to use the terse ones for common tags and the verbose ones for rare ones.

The last time I dealt with XML based services I tried to push "If we can't be right let's at least be consistent"

I've been involved in XML standards making for about seven years, and one of the things I'm most proud of is encouraging the standards body to publish a JSON standard.

XML is extremely flexible, and delivered on being extensible. That is why it has fallen out of favor. XML is pretty unforgiving in terms of what is valid (this is all for good reason) and the extensibility brings with it a lot of complexity. XML is:

- Complex and requires tools with more functionality than, say JSON. Yes other formats do less. That is why formats like JSON are supported completely in most languages and XML and it's dependent standards. For example SOAP is not fully supported by Python.

- Brittle. It is easy to introduce errors in XML that cause most XML parsers to fail. Bad characters, bad or missing escapes, unbalanced tags all introduce points of failure.

- XML is the foundation of even more brittle and complex standards like SOAP and XHTML. Yes, SOAP is really cool. WSDLs when supported are magic. But... when something isn't right, it's very difficult to fix or get working. Meanwhile, a mediocre developer can integrate with a web api that delivers JSON in an hour or two.

While XML can do the job, JSON, YAML and TOML all are a lot smaller in terms of tooling and cognitive overhead, and most importantly, writing a library to handle them is much less daunting than supporting XML and all of it's child standards.

All that said, for more complex data interchange cases, and internal file formats, XML still is hard to beat. So much work is done for you.

Whenever someone mentions SOAP I’m always reminded about this funny article “S Stands For Simple” [0].

———

[0]: http://harmful.cat-v.org/software/xml/soap/simple

It is "Simple" in contrast to things like CORBA and DCOM. Which it accomplishes by not solving the main problem that these distributed object protocols try to solve at all and instead being overly complex RPC protocol with completely unnecessary "object orientation".

On a similar note, the L in LDAP stands for lightweight, which was accomplished by removing advanced authentication features from X.500 DAP and then gradually reimplementing the same functionality with significantly more complex "simple" "framework" that in theory allows usage of any conceivable authentication mechanism, or something like that.

I lived much of that story, and it's only the slightest exaggeration.

I see the main problem of XML in it's model not matching the data model you usually work with. In code you work with arrays or structs/objects that have members. Members can be other structs or arrays or primitive types. Arrays can be composed of structs, arrays or primitive types (sometimes heterogenous, but not very often).

This does not map to DOM at all, in any direction. In XML DOM you have structured nodes intermixed with text, order of nodes is significant (or may not be) and you have attributes. On top you have namespaces.

Indeed, you can restrict to subset of XML and find a mapping that works, guaranteed you will get a very verbose XML that is hard to work with.

I would generally avoid XML, but the alternatives aren't perfect either, which is to say I sympathize to a degree with people who are familiar with XML and dissatisfied with the shortcomings of the alternatives. YAML and TOML are both pretty complex in their own rights--really what I want is JSON with comments and multiline strings for the "human reader/writer" use case. Never the less, I've never been so pained by these deficiencies that it made sense to use XML.

> really what I want is JSON with comments and multiline strings for the "human reader/writer" use case

Sounds like you want json5: https://json5.org/

It also has the "Arrays may have a single trailing comma." feature which is a pain to not have in json.

Oh yeah, I forgot to include the trailing comma in my list of wants. But yes, I probably want something like json5, albeit it would be nice if it had the traction that YAML and company enjoy. One problem with most of these JSON++ dialects is that they tack on a bunch of low-value features like single-quoted strings and unqouted keys that complicate building parsers and facilitate stylistic differences.

> encouraging the standards body to publish a JSON standard.

Were you successful?

Yes.

where can I find it?

hropenstandards.org it used to be called hr-xml.

> YAML is to config syntaxes what Python is to programming languages. Seriously, significant whitespaces?

That's his critique of YAML? That it is too pythonic? You're not getting far in convincing people of something being bad, if your top argument is "It is like python".

It sounds like they dislike the fact that indentation matters. I see a lot of people who get mad about that in Python and YAML has the added annoyance of not allowing tabs.

Personally, I love both Python and YAML for this exact reason (among others). Explaining a hierarchical system using indentation to an average person isn't too difficult, but even very smart non-tech people will get confused by explicitly opened and closed blocks (if-endif or { }). Without proper indentation, the code is unreadable anyway, so why not actually take advantage of that and get rid of the bracket soup that means nothing to you anyways.

It’s not the concept of indentation that’s the problem, it’s the implementation. Tabs vs space, smart editors, auto-linters all make it easier or more difficult depending on the situation, which created insecurities by constantly breaking expectations. In case of code, the fact there is not visible difference (most of the time) between tabs and space make it quite easy for bugs to creep in.

that's why it's spaces only.

If you mix you are asking for trouble, and if you allow to be configured, half will pick one the other will pick other.

So the only sane choice is to just pick one and enforce it.

And by now most of the editors have python rules baled in, so it's not big problem in practice.

By python standard one level is exactly 4 spaces

Right. Also, for those language designers trying to figure out if the right white space is 'tab' or 'space', just do a quick DDG for 'tab' and 'Makefile'. The answer is space.

Are your referring to the fact that Make requires tabs to indent blocks of commands? It's not even whitespace in that case - Makefile treats the character as a prefix. Sounds like it would be even worse if it required 4 spaces...

I actually see Make's use of tabs a great example of why they make more sense for indentation, especially when it matters. 1 \t => +1 level of indentation. No guessing the number of spaces and praying it stays consistent throughout the file.

I don't disagree with you from a language-designer point-of-view: it's an elegant, indentation-agnostic solution.

The problem is that after explaining -- for the 50 gajillionth time -- that "it's a tab, not X spaces", I get a little frustrated.

A valid criticism of YAML is that it requires a very complex parser compared to XML or JSON. E.g. there's plenty of JSON and XML parsers to choose from for each language, but either none or only one YAML parser, and those add a non-trivial complexity to projects.

(XML and JSON are in the same bucket for me, both are simple to create and simple to parse, YAML on the other hand...)

How is xml simple to parse? It is difficult for both humans and machines. Just because there are libraries doesn't mean it is simple. It's a minefield, with security implications.

To be fair... so is yaml.

That's a pretty valid criticism. I am wary of significant whitespaces too as are many people(and one of the reasons I use Ruby for scripting rather than python.)

Yes it is.

Especially after you tried configuring an application which has pages-long comments with commented out examples that are misaligned to indentation, and an app takes few minutes to show you an config error.

It is completely impractical - it looks nice - but we aren't writing for YAML for non-technical users, we are writing it for machines and other technical users.

I seriously wonder why TOML isn't popular.

I think YAML is popular because it has a non-noisy way of including multi-line strings, which are very common in CI /DevOps type things (where YAML is most popular). All of the other features and insanities of YAML don't really matter compared to that.

TOML is good, but it quickly gets confusing when you have object hierarchies / nested tables. I feel like json5 is probably the best option at this point.

That stood out to me too. A good article, but a bit unwise to include a dig at the Python language. At the very least we can all agree it's a very successful language. Beyond that, it's the kind of thing people have strong feelings on. If you want to make the case against a major programming language, that should be done in an article of its own.

I was skeptical too of this sentence, but then I realized that I have never used YAML for files that were more than ~200 lines long. What happens for huge configuration files with lots of sub-trees?

(Comparison with indent-based program programming languages like Python and Haskell is not completely appropriate, because one typically avoids too many nesting levels when writing code. But in a large YAML file you might just need to go deep.)

YAML anchors allow a basic level of reorganization and DRY within your file, not unlike functions in code.

I'm sure that some people will scoff and point to anchors as exactly the kind of complexity a configuration format shouldn't have. But I've personally found them extremely useful and straightforward.

I'm working with a 12k-line YML file with lots of sub-trees right now, and let me tell you, it's a gigantic pain in the butt.

If it were JSON, I could navigate within a subtree by selecting & searching the text between the `{}` delimiters. But instead, I have to gently scroll downwards, keeping a careful eye on the indentation, to see whether I've moved into the adjacent subtree.

A 12kLOC YML sounds like it's always going to be a pain in the butt, but the specific problem you mention should be solvable by any editor that can collapse blocks, right? Most editors I've used (vscode, vim, phpstorm, intellij, even notepad++) have that feature built in or supplied by a pretty standard plugin.

Yeah code folding works for the visual scrub, but it's the "search within" feature that's missing.

An example of my common use case is digging through a Bosh manifest file for a resource with a particular name, then finding, say, the VM type associated with that resource. The name and vm_type fields may be separated by hundreds of lines (it's a machine-generated file that does not retain element order), but they're siblings of each other.

In vim, I could land on the first { and then hit:

  v% (extend selection to matching delimiter)
  /\%Vvmtype: (to search for "vmtype:" within the selection).

In IntelliJ I could similarly extend a selection between delimiters and then search within it.

It's absolutely correct that this is a tooling problem. You could imagine a YML plugin for IntelliJ or vim smart enough to offer a motion for "select all the children of this block," but AFAICT no such tool exists.

Agree that invoking Python is probably counter-productive but I think the basic point is correct. I've banged my head against the wall many times trying to figure out which line in my 500 line YAML document is not indented correctly. Bonus point when Helm generates an invalid YAML document but because it is interpreted, the line number spit out by the parser is not a line number in your actual source template.

Significant white space is one of those ideas that’s very appealing in small examples and demos but falls apart in production. I was a big proponent of it myself until I saw it cause some very subtle production bugs.

Now I’m more than happy to endure a little extra typing in order to be able to defer code formatting in complete confidence to prettier.

This highlights the importance of implementation: this is not a problem in Python because the language prevents those subtle inconsistencies whereas YAML tries too hard to accept anything and has a lot of magic syntax which makes it hard for tools to determine intent.

It's less of a problem in Python but it still exists. A lot of the day to day work of programming is moving and refactoring chunks of code around. Having to pay careful attention to white space the whole time makes this a lot more tedious and error prone than it is in languages without this feature.

Having spent the last couple decades writing Python professionally, it definitely happens but it’s rare outside of people using Notepad. The major programmers editors have been able to adjust indentation on paste for many, many years and, unlike YAML, that will get an immediate syntax error rather than parsing to something unexpected.

I've seen the opposite problem in braced languages. People assume the indentation signifies a block and subblock level, then debug a weird logic flow and discover that the indentation blocking does not match the brace blocking, because someone didn't properly indent after adding a new set of braces. I like python because the parser and the human are both inferring meaning from the same data--indentation is the block.

> I've banged my head against the wall many times trying to figure out which line in my 500 line YAML document is not indented correctly.

How are tags or brackets better here? The "urgh, where's the missing bracket!?" problem has been with us for decades and it's not likely to go away until we move from text editors to structured editors. At least you don't have the problem of misleading looking indentation when the indentation itself is the single source of truth.

XML and JSON are not immune to similar problems, but in my experience you run into it head first with YAML more because when copy/pasting YAML stanzas from one document to another, the indentation can (and often does) get mangled. Since whitespace isn't meaningful you can paste a JSON or XML object as without the whitespace, parse the document and then reformat it to display it nicely. With YAML if the whitespace gets mangled on paste then the document can't be parsed to begin with so the tools you generally use to debug such issues are largely useless. You are effectively back to raw text.

Saying YAML is too pythonic when really you're talking about a file format used by other languages too is a bit cheesy but I get the criticism.

Semantic white space (originally called "The Offside Rule") predates Python.

I believe it was originally invented by Peter J. Landin[1], and yes the name is a joke about football.

The main problem for me is that it makes copy and paste more difficult. You can't just let the editor reformat for you and you have to be more careful with what you comment out.

Python devs seem to live with that OK though (I'm told) so maybe it's not such a big deal.

1. https://en.wikipedia.org/wiki/Off-side_rule

> The main problem for me is that it makes copy and paste more difficult.

Heh, that makes for a good snark for dismissing the white space complaint, it's all from copy-paste programmers! (Or is more properly black space for all you darkmode lovers)

> [...] you have to be more careful with what you comment out.

Not sure if I get what you mean? I think python lacking proper multi-line comments (making things you want to ignore a multi-line string isn't as neat) is an irritant, but I don't get how semantic white space impacts the complexity of commenting?

I didn't like Pythons whitespace when I first started using it after years of Perl and Java. Then after maybe 6 months I went back to writing some Perl code. I moved some blocks of code around, then got the dreaded missing brace problem where they didn't match up and I couldn't which brace wasn't matching up. I realized that I never encountered that problem in Python. That was enough to convince me of the benefits.

Python has the advantage that it is easier to control the level of indentation by breaking things out into functions without compromising the readability of the code.

YAML sort of allows you to do the same using anchors but I find it doesn't help readability as much and sometimes hinders it. I think this might be because in yaml you are describing a data structure and it's far easier to understand that data structure if you can see the whole thing at once.

In python you are describing code and for some reason in my head functions as an abstraction in code make it more not less readable.

This is terrible advice. Exposing any public endpoint that talks XML is a security disaster in waiting and the fact that the author makes no mention of a long history of major security vulnerabilities in popular XML packages does no credit to him.

The "composable" without "confusion about semantics" is a joke, too: in practice it's not even possible to know what whitespace means. Is it semantically meaningful or just there for pretty printing? Also, say you see a commented out region in an XML file. How do you uncomment it? If XML where less braindamaged both problems would be trivial and purely mechanical. In practice you'll generally have to guess.

All in all XML is an extremely mediocre design: it is substantially worse than prior art in countless respects and tremendously complicated without achieving much of anything.

Indeed. Although XML has a grammar its complexity led to countless flawed implementations. The original sin of XML is that it has been over-hyped from the beginning. Is it the cause or the consequence of it being over-engineered, I would not say. Anyway following standards, XLST, XPath, saw poor adoption. I am not fond of JSON and YAML either, but since they are much less ambitious, I am somewhat more forgiving with them. None of them is good at being a config file format. None one them is good at data serialization, that should be binary and TLV encoded.

> it is substantially worse than prior art in countless respects

The prior art being? ASN.1 libraries don't have a stellar security track record either.

Flexible data interchange is _hard_.

XML is basically just a vastly inferior form of sexps, just as ASN.1 at least for the purposes of encoding cert info in TLS is vastly inferior to csexps -- the latter is canonical by design, efficient and trivial to write a robust parser for.

If this is so trivial, why has it not been done? It seems unlikely that nobody has ever heard of a common term.

Uhm, it has been done. Writing an obviously correct csexp serializer is literally 3 lines of code in a high level language and writing a deserializer is only slightly more LOCs (and yeah, I've used it in production for real stuff).

If your model of the world is that because you don't know something it can't be vastly better than the stuff you know and that is widely used, you should consider refining that model. It's not how things work; sometimes fields take a wrong turn that is difficult to recover from or get swamped by morons who drive the good people out. A lot of what we have in 2020 would strike a smart practioner from half a century ago as shockingly braindead and outmoded and a lot would seem like scifi. XML is not a member of the scifi category.

I had whitespace removed by XSLT moment. Why would a format designed for application consumption have insignificant whitespace?

Also HTML whitespace handling makes all sort of bugs in contenteditable. Chrome inserts   instead of space, finally found solution

    * { white-space: pre-wrap }

Still it removes first newline in <pre>, at least that is solved in XHTML.

I would love to see how XSLT can be implemented on s-expressions. Attributes are not required.

One reason to dislike XML is that it can't actually encode text.

This is deliberate, key people involved in XML standardization really hate some of the characters (particularly some "control characters") in text, they don't think those characters should exist, so, they outlawed them in XML.

I know what you're thinking, "Just escape them". Nope. I didn't say you have to escape them, I said they're outlawed, the XML definition says these characters, whether escaped or otherwise, must not appear in an XML document and some XML libraries enforce this requirement from the standard.

Now of course if you know this will happen and you want to sidestep it you can encode all your text in some other way (e.g. as Base64) and then use XML, but wait a minute, the whole point of XML is that it's a text encoding, for encoding text, so this is crazy.

In systems I've built that are mandated to emit XML and yet are expected to handle text anyway, I patiently explain the problem and unless people seem stirred awake and back off on requiring XML, I then convert all prohibited characters to U+FFFD - the Unicode replacement character - and they can keep both halves of the resulting broken garbage.

A "control character" (such as form feed) is kind of by definition not "text", but more importantly this was fixed for XML 1.1, which is like 14 years old at this point.

I have never seen XML 1.1 used in practice. How many parsers actually support it?

Interoperability is exactly the reason why control characters weren't added to xml 1.0.

And

>You are encouraged to create or generate XML 1.0 documents if you do not need the new features in XML 1.1; XML Parsers are expected to understand both XML 1.0 and XML 1.1.

XML 1.1 was considered a failure by W3C and it is not recommended that anybody use it.

Precisely which (printable?) characters do you understand to be outlawed in cdata/pcdata by what XML standard?

For what it's worth, I'd not take a charitable view of this bold assertion in the general case.

You heard of CDATA, bro?

Standard says: "

CData ::= (Char* - (Char* ']]>' Char))

Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] / any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */

Which means the following characters are illegal: #x0, #x1, #x2, #x4, #x5, #x6, #x8, #xB #xE, #x18, #x19, #x1A, and #x1C

I'm not arguing here for or against inclusion of these characters as legal characters in a text block; just saying that GP's point is oblivious to actual encoding (explicit escapes or cdata)

This is an odd criticism. JSON, the popular alternative, also has this 'issue'. In fact, almost any non binary solution, like protobufs, will have this issue, as it's a text, not byte, encoding. The reason for outlawing these characters is that they aren't text, they are, as the name implies, control characters.

Unlike XML you can simply escape control characters in JSON and "almost any non binary solution". Prohibiting this is a weird misfeature of XML.

Of course it can't. It's designed to be human-readable which control characters aren't. What eg. does BEL (ascii 07 IIRC) look like do you reckon.

Ideally it would look like 

Its useful to be able to encode arbitrary characters even if they aren't human readable.

Fair. Is the NUL character included in that?

https://en.wikipedia.org/wiki/Valid_characters_in_XML

"Note that the code point U+0000, assigned to the null control character, is the only character encoded in Unicode and ISO/IEC 10646 that is always invalid in any XML 1.0 and 1.1 document"

That wiki article may be worth a skim.

I'm asking the commenter's opinion, not about current state of XML. Still thanks for the link.

The major issue with XML is all the garbage built on top of it like SOAP, WS-* and horrors like XMLDSIG.

Used purely as a markup language it's ok but you really don't want to be mucking around with stuff like canonicalization at all if you value your time.

> Used purely as a markup language it's ok

But people don't use xml as a markup language; they use it for data interchange and for that purpose it's horrible because it maps very poorly to data structures in most programming language.

Bingo. This is why I hate XML. Nothing wrong with the syntax - I hate that I have to massage arrays, dicts, etc. in cryptic ways to generate the XML output, then massage them right back at the other end. And PHP's SimpleXML module, at least, is anything but - lots of iterators, casting objects to string, and "where did my data go?".

> [...] and for that purpose it's horrible because it maps very poorly to data structures in most programming language.

I disagree with you.

A XML file is, by definition, a [context-free grammar](https://en.wikipedia.org/wiki/Context-free_grammar), the same type of grammars used to specify programming languages.

I will make it clearer with the examples below:

    /* a simple data structure */
    struct Point {
        int x;
        int y;
    };
    
    <point>
        <int name="x"></int>
        <int name="y"></int>
    </point>
    
    /* a recursive data structure */
    struct Person {
        std::string name;
        std::vector<Person> children;
    };
    
    <person>
        <string name="name"></string>
        <vector type="person">
    
            <person>
                <string name="name"></string>
                <vector type="person">
                </vector>
            </person>
    
            <person>
                <string name="name"></string>
                <vector type="person">
                </vector>
            </person>
    
        </vector>
    </person>

To me, it is immediate to see the correct data structure in both examples.

If I had to write the second one in JSON, for instance, I would lose the "template type-information" of the list (vector).

But in practice it ends up any adhoc combination of

<point><x>value</x>...</point>

You can argue this should not happen, that it's not valid XML, etc...but being so verbose and readable, it tends to attract abuse and then hate.

And how is it different from, let us say, the current's sweetheart, JSON? (The comparison is not exclusive to JSON, though.)

JavsScript Object Notation makes a lot of sense to describe... The JavaScript objects.

But, nowadays, I see it universally applied.

As a recent personal experience, I had to use it to serialise communication between a C++ and a Java application---two statically typed languages, with basic types similar among each other but different from JavaScript.

To set the contract between the two applications, I had to write a JSON spec [1], which is written in... JSON. That was an abstruse and abusive use of JSON, in my opinion.

Then comes `jq` and its own notation, in which I suffer to do the basics of a simple XPath---e.g., return all the objects of a certain kind (i.e., a tag) in any level you find it.

Abuse happens [2] because there is a human behind it, especially under the lens of the adage "if all you have is a hammer, everything looks like a nail".

The problem I see in this discussion is that people either want a "magic tool" that works at all cases, or they want to control the human behind it.

In my experience, it is difficult to get both: generality does not play well with constraint.

---

I think most people just need to use a plain-text format [4] for most of the things they want, à lá RFC 822 [5].

</digression>

---

[1]: http://json-schema.org/

[2]: I believe this is a Fibonacci series: https://www.xorpd.net/pages/xchg_rax/snip_01.html

[3]: http://blog.aerojockey.com/post/iocccsim

[4]: https://www.arp242.net/the-art-of-unix-programming/#id290742...

[5]: https://tools.ietf.org/html/rfc822

> To set the contract between the two applications, I had to write a JSON spec [1], which is written in... JSON. That was an abstruse and abusive use of JSON, in my opinion.

The XML equivalent is an XML schema, also written in XML. This is pretty decent as document validation goes.

> I think most people just need to use a plain-text format [4] for most of the things they want, à lá RFC 822 [5].

You linked to formats which don't even support nesting.

If you roll your own markup it's going to be brittle and people are going to parse it with regexps.

JSON has more degrees of freedom due to more literal types and structures, you can build more contrived documents with it.

How so? The JSON model is much more restricted than XML. For example:

* In XML an "object property" can be represented as either an attribute or an element, in JSON only dicts can have attributes. * In XML elements can contain a mix of text and child elements. For serializing objects this is almost never desired. * JSON has a single way to define lists, in XML you need to roll your own.

In JSON "object property" is not synonymous to "dict", dict is just one of many ways to express object properties.

JSON can contain a mix of differently typed objects too, which is similarly meaningless for serialization. And serialization is not the only usage of json.

JSON has a concept of list, but lists can be defined in many ways. And list is not the only collection type one might want to express in json. For example, is package dependency list a list or a dict?

That's one way to map that to XML. But there are a dozen other ways to map it as well, and all equally (un)usable.

> context-free grammar, the same type of grammars used to specify programming languages

As I understand it, few modern programming languages have context-free grammars.

Of course you can serialize a struct to XML in an idiomatic way but a lot of the time you don't. You get an XML document produced by someone else that maps to a horribly non-idiomatic representation as an object in your language runtime. I have personally seen lists/arrays serialized to XML in a bewildering variety of ways. The advantage of JSON I think is that the mapping to data structures is relatively straightforward and idiomatic so when you are creating a type to represent a JSON document you are receiving from some external source, you are likely to get something that looks reasonable.

But nobody marshals like that (except you, obviously)! Your Person struct is usually marshalled like this:

    <person>
        <name></name>
        <children>
            <person>
                <name></name>
                <children>
                </children>
            </person>
            <person>
                <name></name>
                <children>
                </children>
            </person>
        <children>
    </person>

or, if you're unlucky, even like this:

    <person>
    <name></name>
        <person>
            <name></name>
        </person>
        <person>
            <name></name>
        </person>
    </person>

with implicit arrays. Not that your scheme is much better, really, why are you putting you favourite language's types in the tags? My favourite one uses lists and binaries instead of vectors and strings, you know.

Thank you for improving it. In fact, I prefer your first example and if I had thought about 30 extra seconds, I probably would have written something similar.

> Not that your scheme is much better, really, why are you putting you favourite language's types in the tags? My favourite one uses lists and binaries instead of vectors and strings, you know.

Because that was just an example that came up in my mind in less than 10 seconds to make my point.

I am glad that people understood my point and even came up with better solutions to make my argument even stronger.

> I am glad that people understood my point and even came up with better solutions to make my argument even stronger.

Your smugness is unwarranted.

You claimed that XML has an obvious, natural mapping to data structures. After looking at the same extremely simple data structure, two people came up with three different XML representations, each with different pros and cons.

Your argument didn't come out in very good shape from that exchange.

> You claimed that XML has an obvious, natural mapping to data structures.

That is an (i.e. your) interpretation of what I said. Please re-read my text below, in context.

>>> [...] and for that purpose it's horrible because it maps very poorly to data structures in most programming language.

>> [...]

>> A XML file is, by definition, a context-free grammar, the same type of grammars used to specify programming languages.

>> [...]

>> To me, it is immediate to see the correct data structure in both examples.

> After looking at the same extremely simple data structure, two people came up with three different XML representations, each with different pros and cons.

> Your argument didn't come out in very good shape from that exchange.

I disagree with you: both understood pretty well my data structures and came up with even clearer notation, making my point that it is a suitable grammar representation for data structures.

> Your smugness is unwarranted.

Seriously?

You're effectively defining a serialization format using a subset of xml. You can do that with JSON as well, adding "type" properties as required.

The problem is that everybody has their own subset of xml for serializing objects, while for json you can load arbitrary json into a straight-forward tree of dicts, lists and literals.

If receive arbitrary xml the only way to process it is with xml-specific tools because the object model does not map to any programing language.

> But people don't use xml as a markup language;

I do. I use it to build a simple website (well, 'simple' being relative, I guess), and for that it works fairly well.

xml or html?

XML with some HTML sprinkled in

xml alone has a few flaws (empty text nodes) but nothing major

the xml everything culture was a faux-progress (basically language design gone wild)

I recently had to learn about SAML so i read about XML signatures.

That's probably one of the worst concieved security standards i have ever read. It should be in a museum for how not to make security standards.

Honestly though, i don't think its entirely possible to separate XML from the culture around it. XML encourages a certain way of doing things that leads to bad, problematically complex, outcomes.

It's truly a shame that SAML has become the "enterprise" standard when OIDC is superior in every way.

No need to defend XML. XML stands the test of time. Yaml has become the evil it was meant to replace.

If XML had-had a command line structured editor at the beginning and integrated with JS a little bit better, it wouldn't have ever fall from grace.

It pains me that json doesn't have comments, and that so much energy has been expended basically getting JSON to the tooling and conformance support that XML already had.

I like XML, I hand write a ton of XML at work, I like it because I consider it to have some built in sanity checking, its something that is obvious even when using an text editor if its correct or not.

Most editors these days will do similar checking of JSON syntax and a lot do checking against JSON Schema as well if you have one handy.

I also like XML. Getting XML instead of JSON implemented at work is a damn pain in the ass though. I work in GPU research, and the appeal of JSON over XML to a underknowledgable PI is immense. I think it comes from being able to read it pretty easily as a non-technical person.

In what way would you have liked to see the JS integration improved?

I was surprised recently to find out that the browser API doesn't provide a streaming parser for XML, but it doesn't do that for JSON either, so that one at least is a draw.

E4X

Have you checked the API? It was horrible.

I used it in Actionscript 3. I liked it.

The author forgot to mention one the biggest functional benefits of XML which is interpolation: the ability to drop XML components into the middle of text, thereby allowing for markup and nesting of elements in text.

<tag>

  This text node contains <magic>unicorns</magic> and <magic>fairy dust</magic>.

</tag>

There is (as far as I know) no sensible way to add the magic above in JSON or YAML.

>[…] the ability to drop XML components into the middle of text, thereby allowing for markup and nesting of elements in text.

Is that “allowing” for markup, or is it exactly what an Extensible Markup Language was intended and designed to do? I wouldn’t expect a JavaScript Object Notation to be able to do that ergonomically.

XML’s design as a markup language, and not an object or config language, is clear even in that example. The first character of the contents of `tag` is newline.

This example should that XML is decent as markup but for a configuration or data interchange format this kind of syntax is undesirable.

You have text with interspersed elements with the same name, how can this be represented in python other than a DOM?

One of the other benefits of XML the author mentions solves this: if you don't want it, disallow it in your schema.

If I disallow everything I don't want from XML I just end up with a more verbose json that is harder to work with.

That's fine, then XML wouldn't be the proper tool for you and you shouldn't be using it, but I was responding to someone saying that they didn't want a specific thing (non-text inside certain tags) which XML supports.

Right, this, and the lack of typing in XML, are the fundamental differences and the sources of many of the differences in performance, complexity, etc. The question to ask is, what’s the right format for your software’s needs? More often than not, programmatic data structures map more directly to JSON/YAML’s simple and typed fundamentals. That easy mapping plus far better performance are the reasons why people avoid XML these days.

The author of this piece gets it fundamentally wrong in saying that “Enterprise” stink was the reason people abandoned XML. For enterprises that need those features, XML is still a fine format to use. But for most purposes, those features aren’t necessary, and the massive overhead that comes along with them makes it a bad tradeoff.

The old saying is XML is a markup language, JSON is an object notation.

Here’s the thing: each element needs to know what type its children are. Does it contain only text nodes, or can it contain elements? You can’t just put markup in a place that’s only equipped to handle text and expect it to do anything but explode in interesting ways. (“Interesting” as in “may you live in interesting times”.)

The same can be applied to other formats like JSON and YAML: if it contains only text, use a string, otherwise use an array of strings or objects of some defined structure. Here’s one possible representation:

  {
    "tag": [
      "This text node contains ",
      {"magic": "unicorns"},
      " and ",
      {"magic": "fairy dust"},
      "."
    ]
  }

So I don’t consider what you describe to be a benefit of XML at all, except inasmuch as its syntax can be subjectively a little prettier.

Add nesting to the xml and the array of array of arrays starts to look a lot worse.

A good example of JSON stupidity. Square brackets introduce an array, whose elements should be of consistent types. But actually this is not enforced. It only enforces the ordering of elements, which adds quite a bit of information! So this is a list (called an array).

Curly brackets introduce an "object", whose fields are considered unordered. So you are not supposed to have multiple "tag". What happens if this is the case ... unspecified!

JSON is probably worse than XML here.

Is your objection to json arrays only that they are not called lists?

That is one of them. Bad naming is a disease in the SW industry and here is a good example. I also argue the square bracket notation is introduced for little effect.

This is like a strictly worse kind of s-expression. C.f.

    (tag "This text node contains" (magic "unicorns") " and " (magic "fairy dust"))

It is always sad and funny watching the world churn through inferior plain-text-structured-data solutions, while studiously ignoring s-expressions.

Shameless plug: https://github.com/jerome-jh/s_expression

Json is not about markup and shouldn't inject surprise objects into the middle of strings. You wouldn't use json for this.

I would mention date and time. XML has them standardized, see xs:dateTime, xs:time, xs:date. In JSON everyone is inventing their own incompatible representation, can be various ISO string formats, or various numbers (seconds, milliseconds, etc) from some epoch, look at that mess: https://en.wikipedia.org/wiki/Epoch_(computing)#Notable_epoc...

C# has XmlReader and XmlWriter classes in the standard library. Technically they are not event based, but conceptually they’re pretty close to SAX, i.e. they implement streaming forward-only parser, and streaming producer.

C# also has Linq to Xml and the XDocument et al. classes which are a nice way of reading / writing XML.

I personally like XML. It’s a bit on the verbose side, but that makes is way more easy to figure out the position in the tree than it is in JSON. It is also more easily extensible thanks to attributes and namespaces. It supports comments which is mandatory for any configuration format for obvious reasons. It has powerfull transformation engine as well as schema capabilities. JSON schema keep producing new draft of its spec, yet the result is not on par with XML, and more painful to write.

I’d like for XML to make a come back, not as the behemoth it was once, but for case it makes sense, and configuration files is definitely one of those.

> Linq to Xml and the XDocument et al. classes

Yep, pretty sure the OP meant just them when they placed that green icon in the DOM column and C# row. However, the red cross icon in the table says SAX is missing in C#.

> ...everyone is inventing their own incompatible representation, can be various ISO string formats...

If only we had some sort of international body to standardise these representations. This and the original articles "JSON has no comments [and this is how you do a comment in JSON]" undermine the complaint on a technical basis.

String with an ISO standard datetime is perfectly reasonable. And quite standard. Use UTC+0 where possible.

CCYY-Www-dd and CCYY-ddd are part of the same ISO "standard" that CCYY-MM-dd are from

> String with an ISO standard datetime is perfectly reasonable

The standard defines more than one format.

Another thing, because it’s not on the spec, people went creative. It’s too late to fix I’m afraid, some widely used APIs already have these seconds in their JSONs.

The same thing happened with XML, XML Schema and it's datatype support was after the fact of XML popularity, same as JSON Schema, which already solves this for JSON where adopted, with respect to JSON.

While adoption takes time, the fact that you don't even realize that XML had this problem (far worse than JSON with dates) demonstrates that it's not too late for JSON schema to solve it for JSON.

> demonstrates that it's not too late

Back in nighties' and early 2000-s the landscape was different.

Now there're too many internet connected things that aren't a PC (phones, tablets, home appliance, cars, etc.) who run native code which consumes these JSON APIs. Some of these things aren't upgradeable, many of them aren't easily upgradeable.

The problem isn't that there is no standard date time format, it's that there is no standard way to represent the standard date time formats (plural) in a JSON document.

> XML has them standardized,

XML Schema, a separate and later layer on top does.

> In JSON everyone is inventing their own incompatible representation

JSON Schema, like XML Schema, does in fact specify standard time, date, and datetime types and their representation.

There are a couple of things this article gets seriously wrong.

* Front-end. XML is perhaps more compatible to that front end than JSON because an XML document can make use of CSS and JavaScript where JSON cannot. See this example of a language I created in another life and please that document is literally the XSD schema file: http://mailmarkup.org/mail-documentation.xsd

Also, if you want to use either format in the front end you have exactly the same one step: parsing whether it’s using JSON.parse or DOMParser(). Actually, the browser preferences XML here too because the browser can be told to parse the XMLHttpResponse response and thereby return DOM document object. https://developer.mozilla.org/en-US/docs/Web/API/DOMParser

* Failed for two reasons and enterprise is not one of them. XML has some super intelligent features and technologies. The more intelligently you use it the more powerful it becomes. Most people were not using it intelligently. Most people were using it in the most primitive way possible without any advanced features. In that case it was just a bulky text wrapper. Most of the Java configurations were dumb like that.

XML is big. The charm of JSON is that the syntax is tiny. When pushing data over the internet tiny is better. The only way to make up for the syntax bloat was to do more intelligent and powerful things which XML wanted to do but most users refused to figure out.

My reasoned objection to XML, having used it for many years, is that it is unconscionably expensive to parse. At scale, XML is extremely computationally expensive in almost every dimension. It is not a good way to represent almost any non-trivial data structure, but that is frequently its sole use case. XML is a great way to expand your carbon footprint in the data center.

Today, there are very, very few cases where I would recommend XML despite it being widely specified for standards.

While I don't usually reach for XML when I need a data transport format this particular criticism is somewhat unwarranted. SAX parsing of XML is pretty cheap computationally. The problem is cultural more so than technical. Pretty much every language ships with a DOM parser. Not all of them ship with a SAX parser. As a result DOM is the more predictably available choice.

DOM Parsing is both computationally and memory expensive though which I think is where XML gets this reputation from.

> SAX parsing of XML is pretty cheap computationally.

SAX parsing is also very complex programatically. Who wants to switch on various SAX events when handling a XML document? No, we want the full thing parsed into an object.

I've spent a decent amount of time in the XML world and I've only seen people use SAX when performance was a large concern or when parsing enormous XML documents.

> JSON is JavaScript native, while XML is not

Yet we have XMLHttpRequest to thank for the last 20 years of the web.

It is pretty ironic, although XHR does predate JSON.

Crockford says JSON existed since JS was formalized.

He says he discovered it in 1999 but it was in use Netscape as early as 1996.

One thing not mentioned in the article is the lossy nature of XML when coming from data structures with concepts of arrays and maps. A typical array in XML looks something like:

<foo>

  <bar>First element</bar>

  <bar>Second element</bar>

</foo>

which can lead to issues if the second element isn't in there: should bar just be a property of foo or should it be an array? (I'm definitely open to correction on this if someone knows a better way; my experience with this particular issue is from writing a parser to take XML and turn it into a MUMPS-native data structure. I mostly had to have the parser take an educated guess and then let API users specify which elements were arrays if they wanted it to parse the same way against all inputs).

This is analogous to how saving Clojure sets (e.g. #{:a :b :c}) to a JSON document in CouchDB is also a lossy operation because JSON doesn't have sets, just maps and arrays.

Seems like you’re more interested in encoding your implementation details into a text document. This seems like something that could be left to the parser. Provide the target types for the deserialization. This isn’t Python pickle or Java serialization - it’s a text string.

Fair enough. Though wouldn't that work better with languages like Java where the programmer typically ends up making a type or class for most things?

With languages like Clojure it's fairly typical to declare no classes or types and to simply have your data structure be a composition of maps, sets, and vectors, etc.

I suppose you could use your Clojure.spec keywords as the tag names in XML, but that's probably going to be more work than (clojure.edn/read-string (slurp "my-data.edn")). Could be an interesting option if you had to use XML as an interchange format.

Problem is XML is extremely frequently used to represent such types (probably even more so than it's used to represent marked up text). And it's not very good at it.

It's the same problem you'd get by looking at a JSON document out of context ("is this array-like thing an array, list, set? ordered or unordered? type bounds on members? are these integers or decimals with no remainder?") You gotta refer to a schema document or human-readable document or whatever and not guess.

I'd agree that things like JSON and EDN don't convey all the type information that you might want (things you mention like type bounds, orderedness, number format). Though at least in a JSON string like {foo: [elementOne elementTwo]} you know that foo is a list of things, which is something that the XML document doesn't encode for you.

So perhaps I should make my statement more precise and say that XML data, in the absence of the schema information, is lossy compared to some formats like JSON or EDN.

[Edit: to answer the potential question of why I'd be using XML without have schema information, I was being provided the XML by an API that called web APIs and converted the result of the web call into XML (even if it was JSON) before passing it to me.]

I really would prefer to use XML for my next project, but self-censored myself so that if this project is ever open source and has other contributors, I'm not instantly dealing with hordes of users who want to convert it to json for data transfer objects.

Maybe it is because my baby duck syndrome came with xml web services and asp.net web forms, but it is amazing how much scaffolding and engineering work we threw away and are now reinventing. (Wsdl, schemas, etc)

Maybe I don't understand something about xml. As in, best practices. When designing an xml schema, When are you supposed to use innertext, when are you supposed to use a tag inside your tag, and when are you supposed to use a tag property?

First, consider whether someone will ever want to convert to JSON. If so, make that easy.

it's sad that this is downvoted, since it's also true. For example, AWS emits its response as XML, and it's a chore for boto to convert that to a json response. If you don't want to use boto (say you're not in python) and you prefer getting the response in json, you might be out of luck.

Mag property is mostly for metadata, but it does not stop it to use it for this purpose. All three options are valid and you can use as will, xml is flexible enough in this point (unless your xml consumer has some restriction, then follow the consumer spec)

Quite often it just doesn't matter that much. It's a matter of 'taste', as are so many things that are 'designed'. And yes you'll make mistakes a few times. The correct response to those mistakes is to take them as learning opportunities and not dwell on them, instead of blaming and trying to replace the tools.

Of course one can use this exact argument to stave off any change ever...

Reason XML went out of favor I think has to do with decision to focus more on efficient parsing by machine and less on readability and simplicity. There was a competition between XSD (XML Schema Definition) [0] and RelxNG [1] for defining the structure of XML. Personally I preferred RelaxNG, but at that time Microsoft and many big players converged on XSD. So XSD won but XML is out of fashion.

I wrote a XML encoder and decoder with XSD for an EDI application handling 2-3GB files in C++ using MSXML and it was very slow, improved the speed using stream parser, but still slow. Later on using libxml rewrote the parser in C with RelaxNG and the processing time reduced from hours to seconds on the same hardware. At that time as RelaxNG lost in spite of having elegant structure compared to XSD. I had a lot of arguments with my friend that RelaxNG will win because it's readable, simple and support every use case supported by XSD. But due to support from many vendors especially Microsoft (as most enterprises were using microsoft) an inferior solution like XSD won, because Microsoft focused more on computer consumption of format instead of human. So although XSD can be read no one could understand it. This is one of the reason JSON, YAML and TOML have picked up because they offer the readability and parsing at the same time. Hopefully people can understand it this time and do not destroy JSON with complex JSON Schema definitions.

Take a look at both:

   Relax NG example in compact syntax:
     element book {
       element page { text }+
      }

   Relax NG exmaple in non compact syntax:
     <element name="book" xmlns="http://relaxng.org/ns/structure/1.0">
       <oneOrMore>
         <element name="page">
          <text/>
         </element>
       </oneOrMore>
     </element>

    -------------
    XSD example:
    
     <xs:element name="book">
       <xs:complexType>
         <xs:simpleContent>
           <xs:extension base="xs:string">
             <xs:attribute name="page" type="xs:string" maxOccurs="unbounded"/>
           </xs:extension>
         </xs:simpleContent>
        </xs:complexType>
       </xs:element>

[0] https://www.w3.org/TR/2008/REC-xml-20081126/

[1] https://relaxng.org/

There's also the question of using a SAX or DOM based parser.

More than once, I've had to update a program using 32-bit perl from DOM (simple, fast) to SAX (slower but more control) as documents grew and the scripts crashed due to Out of Memory errors. Using SAX gave us event based handling to ignore certain nodes, or cache them to disk in another format as they got too big, and other fun stuff.

Rule of thumb was to multiply the document file size by 10 to get the memory footprint with a DOM approach, because of all the objects, relationships, and metadata that has to be stored for everything. One individual records got up above 100MB on disk we were in the danger zone.

Whereas JSON just loads into hashes and arrays and tends to not pose similar problems.

There's a bit more to it, though. XSD keeps backward-compat with XML DTDs, and XML DTDs, in turn, keep compatibility with full SGML DTDs (XML being specified as an SGML subset). Now DTDs are usually required in SGML unless you're using XML-style canonical markup without omitted tags and attribute names. What makes them required in regular SGML is that element declarations carry tag omission indicators instructing an SGML parser to infer contextually required tags. Tag inference in SGML is deterministic; that is, the parser only considers start-element tags that are required and end-element tags for definitely completed content models at the context position. Also, in the presence of tag omission, an SGML DTD is required even for parsing (for example, HTML cannot be parsed using generic XML rules but needs element-specific content and tag omission rules).

So the design rationale for XSDs to retain backward-compat (what's responsible for XSD's "unique particle attribution" constraint) leads to a different result compared to RNG. By contrast, RNG is a "clean restart" working only for canonical XML-style markup, and is always optional (and only used for "validation").

One could argue XML took away what makes SGML a true authoring format to result in only a canonical delivery format, but these were willful design choices when XML was subset from SGML.

Now as to whether XSD's or RNG's format for grammars are prettier is a moot point. IMO there was never a reason for using an alternate XML syntax for SGML/XML DTD regular expressions/regular content models. Eg compare your example with it's equivalent (simple, compact) traditional element declaration:

    <!element book - - page+>

> less on readability and simplicity

I strongly disagree. The reason XML went out of style is because it in practise was a convoluted complex mess that was difficult for both humans and computers to read.

I like XML (though rarely use it these days) and JSON but I would have to say that the worst XML I've seen was far far worse than the worst JSON.

My biggest problem with XML was using APIs written in Java where it seemed like all the internal classes were just dumped however they came out.

I think the simplicity of JSON forced people to think about how to serialize their data. This Simple serialization made things easy for multiple languages, and therefore it became popular.

The article compares XML with JSON and YAML. That makes sense for some applications, but for other applications, like a structured text, I'd compare it with Markdown. I'm not happy with XML or Markdown for my source, where "source" means what I edit or what I put under version control, which aren't necessarily the same thing. I find it better to invent my own adhoc notation and write a Perl script to verify it and convert it into XML, HTML, Markdown, JSON, ...

It's a shame, but the sad fact is that humanity does not yet have an adequate and widely used way of representing structured text in a computer. XML is perhaps the closest I've seen, but the tools don't seem to be good enough or sufficiently available, and now XML is also unfashionable so I don't suppose the situation is going to get any better.

SGML, of which XML is a proper subset, has had custom Wiki syntax parsing (such as for a fragment of markdown or CSV) since 1986. Agree with what you said about pointless comparisons of JSON vs YAML vs XML, though.

Edit (regarding your point about humanity not having an adequate and widely used text format): we have SGML and it's offsprings XML and HTML. We won't be getting any new formats/standards in these times of platform economy and vertical integration, ever

You ever hear of BNF[0]?

Ugh. I used to write BNF parsers for a proprietary email system in the early 1980s. I’d rather bang rusty nails through my kneecaps.

It was a fairly complete notation, though.

[0] https://en.m.wikipedia.org/wiki/Backus–Naur_form

> I'd compare it with Markdown

Which is a pretty terrible comparison. Markdown allows you to add some structural annotation to your text as you write it, without having to type a lot. That's a useful thing. But textual markup systems like TEI and Docbook go way beyond what you can do with Markdown (also, what can be practically managed in a less 'verbose' system like JSON). They also aren't really intended as authoring tools in quite the same way; you don't code them as you work.

well, you might do some hand-editing of your TEI in some specialized academic archival work, but you wouldn't use that kind of coding for writing a new text.

Does anyone know enough about LaTeX/TeX or RTF to compare them to XML here?

LaTeX is something very, very different. It's based around macro expansion and oriented toward print, not programmatic transformation.

For example if I want a programmatic table of contents for an XML document it's a simple transformation or DOM extraction exercise. If I then transform to XML-FO that will fill in the numbers. LaTeX does multiple full document renders to obtain auxiliary data for a later render to use.

No XML based system does what LaTeX can do for text rendering. It's not impossible they just haven't bothered. What's more impossible is to match the ease of writing text. LaTeX gets in the way much less for most text.

It is not lost. "application/xhtml+xml" accepted by all browsers. Browser DOM is XHTML in disguise

    document.contentType
    //"text/html"
    document.body.namespaceURI
    //"http://www.w3.org/1999/xhtml"

HTML is just a serialization optimized for human authoring, it has its faults

    script = document.createElement('script')
    script.textContent = '</script>'
    script.outerHTML
    //"<script></script></script>"

It may be humanity does not have an adequate and widely used way of plain text authoring of structured text. DOM diff and patch may be lacking as well.

I'm amused because markdown started off with John Gruber inventing his own adhoc notation and writing a Perl script to convert it into HTML ...

XML did many things right and a few big things wrong. It's a tragedy there isn't a single imple, standardized way to define an array through xml as there is through json. The lack of schema validation in json is equally baffling.

A hack for JSON comments

{ "//": "Comment1", ... "//": "Comment2" }

Json-schema.org

> The lack of schema validation in json is equally baffling.

XML schema languages, like those for JSON—of which there are many, again as is the case for JSON—appeared after the format had wide adoption, and are separate from the base standard.

I'm not sure what distinction you are trying to draw here, because I don't really see one.

> IMHO, this argument is shallow, as all 3 formats are text-based. Thus, you can - and should - compress files. Parsing might be a bit slower, but it depends a lot on the exact parser (and the associated technology stack). In the end, the overhead of transmitting and parsing in XML - if any - is negligible compared to the total time in the whole use-case.

That doesn't help a human reader. The XML overhead doesn't matter to computers, agreed, but it matters a lot to people.

> Composable > While XML doesn’t strictly enforces namespaces, it’s considered a good practice. This way, similarly-named entities defined in different namespaces can co-exist in the same document without confusion about semantics.

Sounds useful but it actually isn't. JSON is composable just fine: just put a JSON document as a value in another JSON document (something you can't actually do in XML, you have to strip the headers). By contrast, XML namespaces are this hugely complicated non-solution to a problem that no-one ever actually has: if you write an XPath expression then it might select something inside an embedded XML document instead of the thing you were actually looking for, because the embedded document might use the same tag name. So to protect you from this bug that has never happened, you can't actually write an XPath using the tag names you see in the document, you have to either expand out all the namespaces to their URLs (and good luck with what your XPaths look like now), or somehow "register" all your namespaces with your parser before you parse. It's just stupid overengineered nonsense.

> Several XML grammar implementations are available: Document Type Definition, XML Schema, Relax NG, etc. The most widespread one is XML Schema. Since a XML Schema is also written in XML format, a web server can host it. Then you can reference it by a publicly-accessible URL.

> This approach solves the above issues: when a client receives an XML document, the former looks at the XML Schema URL. It can then fetch it, and check that the data conforms to the schema.

> Changing the data format is as simple as versioning the XML Schema file, and publishing it under a new URL.

And then when your webserver goes down, everyone's programs stop working because they can't load your schema.

Has XML schema ever actually done anything useful for anyone?

Once I've joined a project and there was a problem with text selection in embedded PDF.js

It took some time to find out that CSS rules clashed with PDF.js invisible markup that provides text for selection on top of <canvas> rendered document.

And I am not the only one [1]. Solution? Embed namespace in class name [2] like it is C - gtk_window_new, glBegin, XOpenDisplay.

Yes, namespaces in XPath looks awful, entire plain text XML authoring is awful. XML is application format (hint application/xml), maybe it should not be touched without XML editor.

[1] https://www.google.com/search?q=css+name+collision

[2] http://getbem.com/introduction/

> It took some time to find out that CSS rules clashed with PDF.js invisible markup that provides text for selection on top of <canvas> rendered document.

> And I am not the only one [1]. Solution? Embed namespace in class name [2] like it is C - gtk_window_new, glBegin, XOpenDisplay.

OK, that's a good example. Though I'd argue that in structured data like XML it's much less likely to happen - you wouldn't generally use an xpath that was just "all tags of the given name", whereas applying styling to all cases of a given CSS class anywhere in the document is relatively common. In any case, the cure is far worse than the disease.

> Yes, namespaces in XPath looks awful, entire plain text XML authoring is awful. XML is application format (hint application/xml), maybe it should not be touched without XML editor.

But in that case why bother with a textual format with all the downsides thereof? Something like Thrift or Protocol Buffers makes for much better application formats.

I don't know. XML looks like a hype pushed too far. Like server side javascript, nosql, docker, kubernetes but with a strong support from the industry.

It is really hard to gather useful parts. Entire XHTML push for the sake of it was misleading. Semantic web. Like publish data and they will come. This does not work, data should be crafted with specific consumption in mind.

XSLT as XML makes sense when it is transformed with XSLT. Even than authoring is better with human readable form like RELAX NG compact syntax.

What is meta and what is content defined on consumption point. XML inherited totally needless attributes that just brings confusion.

Namespaces looks like a least of sins. They are useful.

So what is a life in Xmlendom looks like? Documents with slight overhead for human consumption (in browser), just a few clicks to extract relevant data, a few more clicks to make parser, a few clicks to make transformation.

XML plain text editing is as pointless as DOCX plain text editing.

> Namespaces looks like a least of sins. They are useful.

Well, the most immediate concrete practical problems I have when working with XML are a) in the default configuration, a lot of parsers will throw an error if they can't load the DTD over the network, which is an accident waiting to happen b) XPaths written in the most obvious way will silently fail to match anything until you've done a bunch of extra configuration to get the namespaces set up right. Maybe they're not the deepest problems, but the surface experience matters too.

> So what is a life in Xmlendom looks like? Documents with slight overhead for human consumption (in browser), just a few clicks to extract relevant data, a few more clicks to make parser, a few clicks to make transformation.

In my experience anyone who can actually understand those transformations well enough to maintain them would rather be using a real programming language instead of clicking.

> XML plain text editing is as pointless as DOCX plain text editing.

So why not something like protobuf that's easier to work with then?

I am a wrong person to ask. I've never advertised XML as machine to machine format. I've used SOAP, XML configs, it was not smooth.

My interaction with XML mostly was in browser where it shines compared to HTML as application format. I have my own list in defense of XML, wrote first one [1].

Unfortunately your argument is "I don't need it" which is not a lot. Imagine Unicode instead of XML "ASCII works for me, Unicode does not fit in char, not even in wchar, surrogate pairs, access by index, wrong encoding". I mean, sure, stick to ASCII.

Now, transformations, the idea was people would catch clicking quicker than real programming language. Turtle, Scratch have its place. Visual Basic has its place. Excel has quite big place.

[1] http://sergeykish.com/pre-newline-preserved-in-xhtml-test

> My interaction with XML mostly was in browser where it shines compared to HTML as application format. I have my own list in defense of XML, wrote first one [1].

If that's your best argument for XML then I'm thoroughly unconvinced. I can see the theoretical argument for the XML behaviour, but the HTML behaviour seems a lot more useful in practice - a lot of the point of using pre is to preserve vertical alignment, and with the XML approach the first line is forced to be misaligned in the source.

> Unfortunately your argument is "I don't need it" which is not a lot. Imagine Unicode instead of XML "ASCII works for me, Unicode does not fit in char, not even in wchar, surrogate pairs, access by index, wrong encoding". I mean, sure, stick to ASCII.

But there's a compelling argument for using unicode: a lot of characters can't be represented in ASCII, and traditional codepages don't let you use characters from multiple languages at the same time. Where's the corresponding argument for XML?

> Now, transformations, the idea was people would catch clicking quicker than real programming language. Turtle, Scratch have its place. Visual Basic has its place. Excel has quite big place.

But not everything has a place. Some tools are just bad, and point-and-click builders are one such thing IME.

Thank you for feedback. HTML behavior is useful when authoring plain text

      <pre>
    foo
      </pre>

But I write in browser, by content is DOM, HTML is a poor serialization format. Each time I save and reload it eats newline.

Other arguments - </script> inside <script>, </style> inside <style>, <ul> inside <p>. It just works in XHTML, as it would in JSON, EDN etc. HTML parser is a beast [1] [2], you would have much more problems.

World around us clearly states there is a need for a rich text format. Hardly anyone objects web browsers for document consumption.

uBlock Origin "Block element ..." is point and click.

[1] https://htmlparser.info/parser/

[2] https://www.w3.org/TR/html50/syntax.html

who even cares, just use things that everyone uses. XML can be used in a schemaless manner and json/yml can be used with a schema. Eventually the more compact formats are more popular because people can use them for other things such as object dump and actually read it. And a very big config file can be handwritten and code review without too much hassle. It's not about schema or association with enterprise.

I do miss SOAP though. Why are we still writing plumbing code in 2020. And no grpc isn't the answer there is still plumbing code, what's the point?

REST was the answer, because why have a schema define a specific input, when instead it's a lot more fun to make the input come from one of these fun places: Path, Query, Body (Even with a GET - because why not?), Header, AAND let's not forget the Method!

let's not forget the prescribed verbs that give rise to CRUD APIs

To play devil's advocate: why should we have all these weird little files knocking about in /etc, when we could query and mutate everything of interest on the system instead via a Management Instrumentation API?

wrong thread?

Not at all - sort of an oblique argument for the unix way / "worse is better" and reasons why JSON and the disorderly REST-ish array of techniques has come to dominate over SOAP, XML, WSDL, etc.

The unix way: text files in /etc, human-readable, each kind of does its own thing, but it works.

Compare to: WMI, Windows registry, complex NTFS permissions, a bit more unified but ultimately a drag to deal with; complex app servers like Websphere; CORBA; and all the WS-* stuff.

"worse is better" isn't really an argument is it. For earch "worse is better" decision there should be an actual argument for an upside to it, eg: Window registry is more discoverable than a gazillion of directories (arguably). REST and JSON (and WS and XML) were pushed by the web mob who wants all protocols in their stack to be text-based and hackable by inspecting the traffic or whatever. In the dotcom era, their need dictated that of the industry. I think in the next era of cloud and IoT and massive communication traffic we will see more push for more efficient binary protocols (protobuff, avro) which naturally also leads to abstracting them away completely (grpc, thrift), for ergonomics.

I'm not sure "worse is better" is specifically an argument, but it seems to be a decent signal for survival and even broad market success.

Like, if you don't need the most compact and fastest possible serialization, isn't there some benefit in text / human-readable messages? There are lots of tools for dealing with text--searching, filtering, categorizing, and so on. It seems to me that most projects never get close to the point of real suffering due to inefficient serialization, or even due to mushy schema.

Look at Javascript: a terrible language, but very broadly available, and sort of easy to pick up.

"most projects never get close to the point of real suffering due to inefficient serialization"

Individual projects are fine and no you don't personally need efficient anything. But as an industry that serves a wide range of need (people from the 3rd world, embedded systems etc), we do have the need for things that become standards to be efficient, call that lowest common denominator.

Why build efficient cars or batteries if we are currently fine with them? Why go for 5G when I can do everything just fine on 4G?

Could it be because when inefficiency becomes standard it compounds? What if you can have 5G perceived speed right now if all Internet protocols were binary?

> IMHO, this argument is shallow, as all 3 formats are text-based. Thus, you can - and should - compress files.

Not only that, but converting data to text, compressing, decompressing and back from text to data is kinda dumb; there's technologies like EXI (https://www.w3.org/standards/xml/exi) that transport XML data in a compact binary format. It may not be readable in transit, but switching the transport or converting the EXI into text-based XML should be trivial if you want to inspect it manually.

But people seem to be reluctant to have mechanical sympathy, even if the data is barely ever read by humans.

EXI is 'lossless', right? Trivially convertible back to plain XML? Not much downside then. Ideally the downside that it isn't human-readable shouldn't count for much. Technically that's true of HTTPS too. If it's treated as a communication channel, it shouldn't matter at all.

I suppose it depends how good the tooling is. I get the impression it's not natively supported by web browsers.

Is EXI usable in a SOAP service?

XML had its problems, but I wish the baby didn't get thrown out with the bathwater. We got some of the things back (like comments), but e.g. XML namespaces are an amazingly versatile feature when you need to define extensible schemas, or aggregate data from different sources.

XML remains my favourite configuration format, anything that came later just ends up reinventing the wheel, and badly with half of the tooling capabilities available for XML processing.

In the early 2000s XML was like violence - if you didn’t get what you wanted out of it you just used more.... or some horrible analogy like that.

I feel like a horrible analogy is itself the perfect analogy for XML.

Interesting that nobody pointed out the tooling dimension of XML's demise yet. (edit: actually there was another comment too: https://news.ycombinator.com/item?id=24623696)

You really need a high level of support in editors and other tools to make it good for humans to work with, to just showing the AST and to hide the unusably verbose surface syntax. This support didn't materialize widely enough for whatever reason.

Of course you can say that it's just a convoluted way of saying the syntax wasn't good enough - but at the time the argument was that tooling will be good enougg and will solve the issue. So it's still a good lesson about tooling.

In the old days many things that were XML when it should have been JSON, today it is the opposite, many things that are JSON when it should be XML.

> YAML has 22 ways to write booleans - no less!

I would argue that XML, in practice, has many more. Everything in XML is a string by default, and only an XSD can provide actual type information. Lots of things don't, and those that do often define their own boolean types.

> YAML has 22 ways to write booleans - no less!

> I would argue that XML, in practice, has many more. Everything in XML is a string by default, and only an XSD can provide actual type information. Lots of things don't, and those that do often define their own boolean types.

I think the problem happens when there are 22 ways to write them in the standard and you only know 16 of them (or 27 of them). You're going to be surprised (by the standard) at some point.

If the standard doesn't define anything, then you at least know the responsibility for the definition is between you and your integrations, so at least you won't be surprised by the standard itself.

As far as standard representations go I think the ideal numbers are 0 and 1: either it should define nothing or pick one unambigous standard representation.

Ten years ago, I loved XML mainly because of what I could do with XSLT, and I wrote many websites where the data were XML (static files) and the XSLT scripts acted as the views to output HTML. All that was server-side (ASP or PHP). The equivalent for me today is JSON for then data and the Lodash library to transform them. The environment has switched to JavaScript (Node.js + browser), which was a tremendous improvement. For me, Lodash is the new XSLT - all about data transformation - although they're conceptually unrelated.

I so agree with this. I have been complainging in different companies that using REST is not practical between application (at least use JSON-API then). REST is usefull for frontend <-> backend communication. SOAP was horrible simetimes but it worked. If you change the contract the clients simply do not compile. This is not the case when you change a service using REST (yeah there are some fixes for that but they are not common and widely used).

> I have been complainging in different companies that using REST is not practical between application (at least use JSON-API then).

JSON-API is REST, though not all REST is JSON-API.

> REST is usefull for frontend <-> backend communication.

Actual REST is least useful for communication between a single frontend and backend controlled by the same party.

> SOAP was horrible simetimes but it worked.

More accurately, SOAP was horrible and it sometimes worked.

> If you change the contract the clients simply do not compile.

I’m not sure how that would be a good thing, even if it were true, which it isn't unless you are fetching the WSDL and then recompiling clients each time you try to access the server.

> This is not the case when you change a service using REST

Which is a good thing, HATOEAS plus resources described with media types and using content-type and accept headers properly, all of which is part of REST over HTTP, give you plenty of opportunity for clean error for a representation change you can't handle without requiring you to try compiling at each access.

> Actual REST is least useful for communication between a single frontend and backend controlled by the same party.

I know I'm going off-topic but what is best way to communicate between frontend and backend except for REST? I know about GraphQL even though that is not widely adopted yet and in a lot of cases never will be.

Just as an anecdote, I recently had to generate some XLSX which, even for XML standards, is a horrible standard and the only library that seems to be able to accomplish this (for Java) consists basically of some autogenerated classes from the XML schema itself that have zero documentation.

But the worst part was when I tried to add an integration test for it. I tried comparing the generated file with a fixture file, but that didn't work, because XLSX are zipped XML files and zip isn't deterministic. But even comparing the two file directories of the unzipped files and then comparing XML files pairwise didn't really work, because apparently, XML can be serialized in different ways and it turned out that my CI somehow serialized it differently than my machine (somehow the namespaces were handled differently - not sure if it was an OS issue or some transitive dependency issue). In the end I was only able to solve it by using another library that can specifically diff XML files.

if you smash the file permissions and dates on the file it can be deterministic.

Maybe (didn't look into that), but that wouldn't have solved the "somehow, XML is serialized differently in some cases because of namespaces" issue which, honestly, I didn't even try to understand because it seemed like a pointless rabbit-hole.

Indeed there's generally no guarantee that two XML serializers will choose the same namespace aliases, or handle ignorable whitespace the same, etc.

You gotta think of XML as an object graph (and not a bunch of tags and text nodes) to have any joy at all with it. Unit tests should not compare the serialized bytes, but rather deserialize and/or event-parse and compare the graphs or events.

I'm not suggesting any of this is pleasant, but, there it is.

Thinking of it as graph helps due to its many branches, but it's still hierarchical.

It's actually an arborescence which is a kind of directed acyclic graph (it's a directed graph in which there is exactly one directed path from the root to any of its children).

Yep, I ended up using https://www.xmlunit.org/, which seems to do the job well.

Nowadays there is a canonical serialization for XML. Not sure if all tools support it though.

I don't disagree with every point the author makes — and I cry inside a bit every time I see YAML used for serious work, with data loss and outages caused by that ambiguous — but I think it'd be useful to focus on why XML became unpopular. Java and SOAP culture had a role there but most of it came down to two things:

1. The user experience sucked

2. The developer experience sucked

What XML needed was good quality editors, validators, programing language libraries, and, above all, documentation for people who aren't on the standards committees. What we got instead were people on standards committees and enterprise architecture groups trying to show off what deep thinkers they were, with the assumed inevitability meaning that boring details like actually implementing something could be left as an exercise for the student.

Namespaces are a great example: this is a core language feature, a really powerful tool, and you'll hit the laziest possible implementations as soon as you try to use them. You start with something reasonable:

    <foo xmlns="http://example.org" xmlns:custom="http://example.org/somethingelse">
        <bar>
            <custom:baaz>…</custom:baaz>
        </bar>
    </foo>

That's a nice way of separating ownership, looks great. Then you start trying to parse it, write XSLT or XPath, etc. and find that you can almost never write “custom:baaz" the way it shows up in the document, or often even /foo/bar, because the tools expect you to work with the internal representation (“{http://example.org}foo/{http://example.org}bar/{http://examp...) and so you might see error messages saying "<custom:baaz> not found” in a document which has that exact string, validation errors from tools which use different libraries than what was used to author the document, or, better, silent data loss because you didn't notice that some selectors weren't matching. Many programming language interfaces will require you to repeat those declarations everywhere because, of course, it would have been far too much to expect that a query against element <bar> inherit the namespace declaration from its parent.

Again, this is a key feature which has been around since 1999 and decades later you can still still find people writing things like //*[name()='baaz'] or stripping namespaces entirely before rage-quitting XML. Everything harder is worse: basic usage requires internalizing 6 different ponderous specs on different release cycles, figuring out which of the examples had never been tested and which worked when the author wrote them but are now unsupported by the particular tools you're dealing with, etc. Bonus points for realizing that you had a never-valid document which was just punted down the line by multiple enterprise systems which have had increasingly lenient validation added over the years by developers who were told it was too expensive to fix the previous system.

I'm not sure XML can be saved at this point but I'd start by modernizing libxml2 since half the open source world is stuck in stasis when that stopped getting new features in the early 2000s, and then focusing on non-expert usage. A full GUI editor would be a lot of work but simply having high-quality command-line tools and implementations for common programming languages would go a long way towards not giving people excuses to switch.

Here's a fantastic article about the various misuses of XML that I greatly enjoyed: https://www.devever.net/~hl/xml

Yeah well just try to use XML as your config format and spit JSON out of it for your frontend, if you are using Java you'll quickly realize how obsolete and inelegant is this horror that they dare call JAXB.

XML specified with a mix of RelaxNG and Schematron is a really nice pairing. Both are interesting and a little different to the usual schema languages.

RelaxNG is like "regular expressions for trees". It has a compact syntax like this:

    start = element library {
      element book {
        attribute title { text },
        element page { text }+
      }
    }

.. so books in this library (root element) should have a title attribute and one or more pages. There's a (pretty geeky [] :-p) description of the algorithm here [1] (: Brzozowski derivatives + Haskell!).

Schematron is another schema language that allows specifying constraints as a series of pattern rules:

    <schema xmlns="http://purl.oclc.org/dsdl/schematron">
      <pattern>
        <rule context="book">
          <assert test="let $b := .[starts-with(@title, 'A')] return $b/page/count() >= 100">
            Books whose title starts with "A" must have 100 or more pages.
          </assert>
        </rule>
      </pattern>
    </schema>

So it allows specifying constraints that would be too tedious or impossible with RelaxNG, and allows keeping the RelaxNG grammar simple.

The test on the Schematron "assert" element in the example uses "let expressions" [2], one of many additions that makes XPath and XSLT 3.0 feel a lot like any other modern functional programming language (other features: higher order functions, maps/arrays/JSON support, and more). This presentation [3] shows a few more 3.0 features.

If lisp is for lists, then XSLT definitely is for trees! I suspect as people start forgetting the baggage of SOAP and XSLT 1.0, the language is going to become more and more popular since it is so useful. BTW, there's now a JavaScript implementation that runs on node and the browser too [4].

1: https://relaxng.org/jclark/derivative.html

2: https://www.w3.org/TR/xpath-30/#id-let-expressions

3: https://www.kosek.cz/xml/2019xmlss/Kosek_XSLT4DailyCoding.pd...

4: https://www.npmjs.com/package/saxon-js

I don't see a reference to Siméon and Wadler: "So, the essence of XML is this: the problem it solves is not hard, and it does not solve the problem well." http://homepages.inf.ed.ac.uk/wadler/papers/xml-essence/xml-...

I think the problem with XML is that people tried to make it into something as comprehensive as CORBA (this is SOAP) so that all the decisions about the communication was done in-band. JSON requires everyone to make out-of-band decisions about what the JSON should look like: people talking to each other, imagine that! The former seems quaintly antisocial nowadays.

I don't care about anything regarding XML, except the fact that it can read files and make http requests...

XML was designed for documents, not data. By document I mean large amounts of text. It came out of the publishing industry. In that context, the tags end up comprising an insignificant portion of the total document, so is not overly verbose.

It's too easy to produce some abhorrent XML if you don't put much thought into it, straight from tables and objects. And then people are surprised that what they see is revolting, hard to work with and reason about.

XML is in a totally different category than JSON: it's a tool to craft formal languages. A data format or a config is also a language of sorts, so it can do that as well, but it's real purpose are languages in general. XML languages are what is left of a language if you omit parsing and start with AST.

Configs and data formats normally don't require all the benefits of a language (e.g. composability), but other things do, e.g. UI systems.

XML is great for interfaces, ain't it?

Surprised that nobody has brought up size over the wire yet. XML, by virtue of the fact that it has both opening and closing tags, tends to be more verbose than other standards. With larger requests, this can translate to hundreds of extra kilobytes transferred. Even for smaller requests, people in bandwidth-constrained environments will probably notice performance improvements when using a more concise standard over XML.

Especially for repetitive files, I did some testing for an ETL process that was effectively "here's multiple CSVs in an XML file" - break that into multiple csvs (which matched the customer expectations anyway) and got 30x savings in size.

How much of this is effectively making a custom compression scheme? Would you get better compression by using the gzip compression of HTTP?

That's a fair point, but I am talking just zip compression comparisons, not http actually.

XML's verbosity is solved with compression. Your webserver is configured to compress text-based formats, right?

I think his notes about performance miss the full picture. Obviously gzipped XML and JSON aren't going to cause network slowness, but I've been on big projects that use XML as a data interchange format and XML marshalling and unmarshalling with validation was a bottleneck in the system. We used JAXB and loading the hundreds and hundreds of schemas in JBoss was so expensive that they had to bump up the memory to 32 gb on our personal computers. In the end XML schema validation had to be turned off (removing the whole point of it). Also, this was Java which is compiled and obviously very fast. Can you imagine the same thing in PHP or Python? It would be impossible. I'd be curious if the author ever did a large XML project in a dynamic programming language. I seriously suspect not and these languages (PHP, Node, Python) are the most popular and productive languages in the world.

XML is good at some specific things, but as a general use data-interchange format it is dead as a doornail and there is no good reason to bring it back.

Most common XML parsers in PHP uses libxml which is written in C, so it should be relatively fast, however I haven't used it for super large XML-files and not for every request. It would be interesting to test.

The main problem with XML is that writing a safe XML parser is difficult. But writing, say, a safe JSON parser is not much easier; writing a safe ASN.1 parser is much harder. So it's not clear whether there's a better possible declarative format.

Writing a safe tagged netstring parser is pretty darn easy, for many definitions of safe.

There's always the issue of very deeply nested data structures causing lots of allocations, but no nested parser can avoid that.

Full featured XML parsers can read arbitrary files from disk, which is usually not what you want from a data interchange format.

Gah. I dislike XML, but I had forgotten how much I loathe ASN.1.

XML is hard to parse because they mixed compression concerns in with encoding, which is a security nightmare. What’s hard about JSON? There was some looseness around the spec for certain characters, but overall, it’s pretty straightforward.

Reply on news.ycombinator.com