Parsing XML at the Speed of Light (2012)
aosabook.org
aosabook.org
I mean the last thing I heard from XML was EXI, a new way to transfer XML documents; instead of encoding it into an ASCII representation and gzipping it, it instead (iirc) turns an XML document into a binary representation or stream containing events similar to SAX as mentioned in this article. That could then be reshuffled and compressed for further efficiency. It was a way to improve transfer performance by a lot at the cost of human readability on the line, but, nobody should read XML documents on the line and they would probably use a client (that understands EXI) anyway.
The only cases where XML tech has survived is where it got in early and inertia has kept it there and because it was "good enough".
I genuinely dont know of anybody building new technology on a foundation of XML. There's always something simpler, with a lower bug surface that will handle the job better - even in its originally intended domain of document markup.
I am currently building something new that uses xml for a properties file. My use case requires sql inside a properties file (without the use of additional .sql files). Really wanted to use json/yaml because I felt like I was in the 90s, but xml was just a lot better for this case.
This: |
Always "seemed" a lot cleaner to me.
JSON was never intended primarily for user readability. It's for sending data over a wire. Its focus was on parsing simplicity, ease of serialization and deserialization and to be incidentally readable.
<entity attr="whatever">Try to make this <appearance>look good</appearance> in <markup>YAML</markup></entity>
As I said, horses for courses. FWIW I actually like YAML for certain limited use cases, just not where XML is better. And vice versa.
entity:
attr: whatever
text: |-
Try to make this *look good* in **YAML**.
Markdown is surprisingly effective as an XML document markup replacement.
What do you mean by this "data defined by a second schema inside a first schema"? And why is it a good idea?
XML has tools to deal with this situation, where it knows that what's found inside <picture></picture> can be evaluated according to the SVG spec.
You also have namespaces, to deal with that a <title> might be a thing both in a text document and a song, and you may need to use both inside the same document without conflicting.
How, exactly, is JSON with JSON Schema inadequate for that?
The ability to design a schema that could be passed around to let people validate the data and structure of an XML file before sending it was really slick. The schema designer tools were easy to use.
I actually enjoyed using WSDL based web services specifically because of this. You could take the file, which was just a schema and generate all the code and data structures you needed to interface with the API from any language…and you got the ability to validate your data client side before even bothering to send it.
It definitely had some pain points, but overall these were the good points that I remember.
GraphQL is probably the closest next evolution.
Tooling around JSON Schema seems to be a bit lacking though. Like, Swagger UI and editor doesn't fully support the latest JSON Schema version, especially w.r.t. choice-like oneOf elements.
And if any of you know of any good tools to visualize a large JSON schema, similar to XMLSpy, that's compatible with the latest JSON schema version then I'd love to hear about it.
If you have a problem solved by XML and not other technologies, you can still use XML.
Make sure you have thick skin. Your peers will laugh at you. And your colleagues may veto your choice.
How would this compare to protobuf, in terms of efficiency?
{"markup": "<div>The <a href=\"https://www.json.org/\">JSON format</a> was invented by <em>Douglas Crockford</em>.</div>"}
Most JSON is consumed in the browser, which also has an XML/HTML parser engine sitting alongside the JSON one.
There are people who hold the opinion that "Everything XML wanted to be, JSON did better." <https://news.ycombinator.com/item?id=28667089>
(The fundamental idea of the parser was to use ideas taken from Warren's Abstract Machine for Prolog)
It’s downfall was really how broad it ended up being with way too many ways to represent data structures - JSON was refreshing with it’s limited vocabulary.
Before I did this the previous engineer had written a bunch of stored procedures to do this that was both brittle and slow (processing time was 11 hours) but the SQL was fairly decent.
Once I used XSLT to do this the task ran in under 90 minutes and it was far easier to maintain (new fields, changing columns, etc).
The allocator section is particularly clever and shows just how precise and utilitarian the design is.
Ironically, I’ve only ever ended up parsing a handful of XML in my life, but it’s reassuring to me that people will look into these algorithms and see how beautifully crafted they had to be in order to go so fast.
How? I would violently object for implementing an in-house XML parser as opposed to just finding a way to include libxml2. It's under the MIT license, so I doubt that was an issue.
https://github.com/ccxvii/snippets/blob/master/xml.c#L326
Or in lua with ~200 lines altogether:
With a thirdparty library you have to follow security updates (also to parts you don't need), may have build issues on some platforms you want to support (something embedded?) and so on.
If you don't need XML, but just a few xml-like tags a custom parser can be done quickly. But yes then you get to the NHI problems and what if your assumptions about XML subset were wrong ...
Personally I would lean on an XML lib (maybe libyml2, maybe qxml, maybe windows API, ... depending on context) as XML is complex, but generally adding libraries introduce their costs.
https://patents.google.com/patent/US9110875B2
The hardware was an FPGA-based PCIe board, but the design also ended up in a custom CPU.
Here's one thing I remember: namespace prefix declarations are applied retroactively within XML elements. This is not easy to deal with when trying to design streaming hardware..
The added complexity is in a stark contrast with the namespaceless XML core, where all parsing decisions require a single character lookahead and you can report items as soon as their data is ready. It's interesting to think that a slightly different syntax for namespaces would make these problems non-existent. E.g. if namespace prefixes were at the end of a name (template:xsl) and namespace declarations were standalone elements enclosing the rest of the XML, it could be much simpler.
What's the problem?
Edit: if you mean be in the extremely rare state of having to write an XML parser, it's very straightforward.
Where is your engineering, inventor spirit? This seems snowflakish to me. You want everything handed to you on a silver platter. But the reality is there’s lots of data out there that has no schema and never will. And if you don’t have proper tooling, you’d quit? Why not MAKE your own tooling?
Oh I meant to say I feel the same the way as the OP ('similar sentiments'), but I wouldn't literally quit my job. You're right, there's always going to be data out there with no schema; CSVs are one example. But if in your day to day work you're constantly encountering data with no schema or the structure/format/schema is constantly changing, then chances are you're dealing with a crappy system or working in a crappy job.
And making your own tooling is fun and all for the first time, but there comes a point when after you accrue enough programming experience, you realise how redundant it is, especially if the data is in a format as old as XML for example. There's a massive amount of tooling around XML, from XSLT, XQuery, XPath and all of them have facilities for dealing with schema-less XML data, but if you were to use a schema (or the vendor/3rd party provides one), then 90% of issues (and sometimes even code) dealing with schema-less XML go away. And the same is true for JSON as well.
I find it incredibly funny that people heap scorn onto XML as being some horrible legacy thing to work with, but then will go on happily working with JSON/YAML/or whatever and then have to deal with ALL THE EXACT SAME ISSUES, all the while not realising XML-related tooling has already solved them.
Until 2020 I worked in data exchange between an online retail company (think Amazon in the good old days) and its suppliers.
I wish people used XML more. Not because it's cool or practical or any of those reasons.
It's because 90% of non-tech-savvy people still do their IT work by someone manually filling in cells in an Excel worksheet.
I do understand your attitude - but I think the world would be served a lot better if we accepted formats that were viable, just not en vogue, if it at least got rid of all the crap we are using instead.