In Defense of COS, or Why I Love JSON and Hate XML | Hacker News Reader

In Defense of COS, or Why I Love JSON and Hate XML

jimpravetz.com

44 pointshoov45 comments

XML, being in the markup family tree, has a lot more history then simple json encoding ... measuring its usefulness on a corner case has always been well ... boring. I am glad people are using JSON to sling simple data across the web versus markup.

Come back to me when you are using json to encode an entire document ... you might look at XML a bit differently.

tl;dr use the right tool for the right job.

"Come back to me when you are using json to encode an entire document"

How is that relevant to the article, which is about COS? In other words, what does COS lack what XML has?

I could just as similarly ask why the author even mentions XML ?

* Infrastructure (schema support - DTD, Schema, RelaxNG; transformation - XSLT) * No obvious document format (What encoding are the strings? How to escape characters? * Only used to describe predefined object types (boolean, strings, arrays, dictionaries * Hard to ensure the integrity of the data without interpreting the data from the interpreter itself (no external validation)

You know this can be done on top when you have demand for this? I prefer a non-bloated protocol format over XML anytime. How often does the DTD not matter at all ? How often is the encoding fixed by convention ? ...

I use schemas (in the form of RelaxNG most of the time) almost every time I deal with XML. Together with schematron you can make very complex lint-like scripts to verify your data. Actually I program in XML with a self created programming language (formulated in XML). This together with RelaxNG and a good XML makes it fun writing XML and absolutely (syntax-)error free.

Yes,

But I would claim that by being designed for many jobs in a sloppy way, XML became a terrible tool for all jobs.

Json is a good replacement for XML in some of the application for which some folks foolishly targeted XML (I worked on a server back in the day that really did process five times the data 'cause of our use of XML for interchange - as was the new standard at the time, remember "XML everywhere!"?).

Html is a good tool for web documents (who would have thought?) but you're right, XML fills in for a lot of other document uses - XML is so far the best generalization that splits the difference between a word document and an HTML file. But I suspect the celebrate if someone could put forward a better such generalization because even in the realms where XML is the best tool available, it is a bad tool.

Perhaps if more people admitted the awful attribute/value ambiguity problem that the article very intelligently calls-out, the use of XML would be less painful. If we called it "Inconsistently Structured Data Intermingling Format" ISDIF, the young developers would have some idea what they were getting into.

'XML became terrible for all jobs' ... you do realize that there are literally hundreds of billions of xml encoded documents out there, happily doing what they are supposed to do ... not trolling; but lets put some of the comments in context for what they are.

XML in its original role of extensible markup is thriving and completely successful ... as I previously mentioned, I am glad we are not slinging around angle brackets and happily use JSON instead.

I agree that XML was hijacked during its hype cycle to do a lot of jobs it should have never been intended to do ... AJAX (see the X) was a side effect of this, and we moved on to AJAJ ... evolution sometimes needs different routes to get out of local maxima.

Note also that there is a very long tail of XML vocabularies that you will never use or hear about that get extended and reflect their authors intents, w/o nary an agreement required between you (or I) to get real work done.

HTML5 though is where I have the problem (in terms of XML Failure)... baked in controlled vocabularies ... hmmm, what happens when your tag (or attribute) du jour doesn't pass muster with the WHATWG ?

I console myself by saying that both XML and HTML5 are part of the same markup family, just a short term family dispute for the time being; never bet against markup (or data for that matter) as they tend to stick around a lot longer then the programming languages that generated them.

It's a shame AJAJ is harder to pronounce than AJAX. We need a new term CEOs can learn to say. Then when they start demanding we use this latest technology, we'll be ready.

Feel free to excise XML from any buzzword by claiming the X doesn't stand for XML, it's a free variable. "Our X is JSON"

I sure do. I perhaps should have conceded that once you get up to the level of Large Enterprise Monstrosities (LEMs), you have something almost by definition "terrible" (or at least messy from the viewpoint of smaller, more coherently architected systems) and thus in that situation, I might not, maybe, have any basis for criticism.

But still, one thing I'd speculate is, that that one attribute/value ambiguity problem just might be sooo bad that even in the realm of whatever-monstrosities-that-have-meld-together-messy-stuff, XML would do better replaced by a different whatever-monstrosity.

hehe, I like the denotation LEMs ...

The lack of user-defined tags and attributes (ignoring data-* for a minute because it's different) is what makes HTML such a more pleasant document/markup language than XML.

XML schemas are hard because allowing people to define an ad-hoc ordered hierarchical parsing structure is hard, so most people don't do the schema part (or ignore the schema in the real world) resulting in ambiguity in the rules about what sort of constructs are allowed in the document or what they mean, resulting in XML formats that aren't really interchangeable.

HTML5 relieves this by having only one markup format that's actually specified with a real common understanding instead of a multitude of formal and informal markups. Evidence that the world needs more than one markup format is thin on the ground, what most people need is the ability to locally distinguish between and identify things, and class and id are complete and minimal for that job.

the lack of being able to extend or define my own tags and attributes feels like a reduction in freedom to me ... but I guess it depends on your perspective. If you feel comfortable with browser companies and a small handful of ppl defining a rigid vocabulary for the world to use, that is your option; me ... I don't feel comfortable with that situation.

XML schemas are hard ... the xml suite of technologies certainly did not get it right the first time around, but slowly those 'hard' technoloiges get eaiser to use (either because of tooling or in XML Schema case we now have v1.1 with things like assertions that make validation a lot more useful and easier). In the meantime, I am certain that we will see every XML technology eventually will get regenerated and replicated within the JSON stack of tech.

Lastly, I agree that having a sanctioned vocabulary called HTML5 is a 'good thing' ... however saying that the world only needs one markup format is plainly incorrect, its akin to saying we only need a single spoken language; sure a lingua franca would make life (and processes) a magnitude simpler but hoping for this situation to actually occur is a real 'pipe' dream.

IMO, if a 'fallacies of data' type edict was to be handed down, then 'planning for homogeneity (lingu franca)' would be in the top 7.

Dings XML for crappy commenting syntax then gives JSON a pass for not supporting comments at all.

I love JSON, but it does have it's issues.

On the other hand, comments were explicitly excluded from JSON by Crockford (with the thinking — probably correct — that they'd be abused to embed such things as parsing directives or other out-of-band content)

I believe he was correct at the time, he removed them once people started using them for parsing directives. (JSON hasn't always been frozen)

Having seen/used the rudimentary commenting mechanism in Wavefront .OBJ files (a 3D interchange format) to stuff in more modern information (tangents, additional UV channels, etc. etc.), I can say that comments can help a format live on well past its expiry date.

That said, I will not pretend for a second that this is a good idea. Crockford did the right thing.

JSON not having comments is a benefit when used for it's primary purpose: information exchange. JSON is not a great format for configuration files or static documents even though it's increasingly used for it.

YAML is better suited for configuration files.

Yaml is not good for configuration files because it is not easily human-editable. It seems "easy" but meaningful whitespace is a cluster-fck.

Edit: I have been looking for reasonable configuration file formats for a while. Json actually has pretty bad human-readability at any scale because of its quoted key-values. Yaml is easily readable but when a user tries to change anything, things go to hell. The humble ini-file has won so far. It's limited and not fully standardized but it is human readable and human writable. I'd love see something better but human readability/writable is going trump all sorts of cleverness.

Edit2: From http://en.wikipedia.org/wiki/YAML "The specific number of spaces in the indentation is unimportant as long as parallel elements have the same left justification and the hierarchically nested elements are indented further."

Yeah, a user of your software is going to be able to understand when they mess up on that rule, riiiight. Screw Yaml.

I rolled my own format specifically for configuration. No significant whitespace, delimiters may dangle, and the parser implementation can be configured to accept very high ambiguity(including, if desired, mixtures of sequence and key-value data).

https://github.com/triplefox/triad/blob/master/dev/com/ludam...

A "real-world" example https://github.com/triplefox/triad/blob/master/examples/Asse...

  It seems "easy" but meaningful whitespace is a cluster-fck.

Tell that to everyone using Python. If your editor can't handle meaningful whitespace transparently, it sounds like you need a better editor.

Python is a programming language. Worlds of difference there.

you need a better editor

I think you misunderstand the concept of "configuration file". These are not intended for programmers but for users of the program you write.

White complaints about Pythons' meaningful white are still common, programmers can get over that by, indeed, getting a better editor or otherwise wrapping their head around the situation.

But end-users can't be expected to have this kind of sophistication. Explaining to someone while things they can't see have made their program usable is no fun.

I wish there existed a popular config format that was somewhere between YAML and JSON. In my current nodejs project I sometimes use js files for config because js supports comments and is more lenient on syntax then JSON. I prefer using { } to group objects, rather then YAML indenting, but otherwise am okay with YAML and only don't use it because the parser is not part of Javascript or nodejs.

I don't know that it applies here, but sometimes it is better to not have a feature at all than to have it badly.

Explicit is better than implicit. If you have to explain something you've probably written the JSON wrong.

You can always do this:

    {
        'people' : 10,
        'desc': 'People who will attend'
    }

>Explicit is better than implicit.

Which is why not intermingling comments with data makes sense.

>If you have to explain something you've probably written the JSON wrong.

You could say the same about code: "if you need to comment then your code isn't clear enough". In reality sometimes commenting code or data is helpful.

JSON not having comments is a deliberate design decision (a feature if you want). The reason is if it supported comments, they would have ended up being used to for meta languages and parse directives that would have created JSON documents that could not be parsed or processed by all JSON parsers -- it would have fragmented the JSON ecosystem quite a bit. So it might seem like an accidental bug or omission but it is not it is on purpose and I agree with it.

Like many, I feel that the poster is throwing the baby with the bathwater. Yes, SOAP and XML Schema are horrible. Don't use them, then. Yes, XML is verbose, but that's exactly why Relax NG has a compact syntax. Use that if you don't like the XML syntax. Yes, data can be expressed as attributes or elements, but there are simple rules of thumb to decide between one or the other: if your data can have structure, or you may want to have multiple instances of the same thing, it's generally better to use an element; otherwise an attribute should do the trick.

There are also errors and approximations: XML did not introduce the bracket syntax, it inherited it from SGML. A DTD is not a schema (and if you want to criticize XML, you should point out that it should not have inherited DTDs from SGML.) He doesn't even mention the worst part about comments, which is that you can't have -- inside a comment (very annoying when commenting a large block of data...)

XML has many beautiful applications, like SVG, SMIL (which never took off but keeps getting rediscovered/reimplemented in an inconsistent manner [full disclosure: I participated in the SMIL and CDF W3C working groups]), XSLT, &c. XHTML was not perfect by a long stretch but the new HTML5 syntax is much, much worse.

Use XML, JSON, and whatever is necessary to get the job done. For the project that I am working on right now, I am using XML for serializing Web app descriptions; in this situation, XML is clearly better than JSON.

the main arguments are about syntax. there's no mention of namespaces and schema are dismissed because the author didn't use them. no mention of tools for automated processing.

this is not a very good article, in all honesty. he doesn't like the syntax, but doesn't seem to consider that different technologies can be suited to different problems, or that he simply hasn't experienced the kind of uses where xml works well.

Agreed. I like JSON and I think it's the better choice for serialization and data exchange in many cases where XML is used today, but pretending it's the magic alternative to anything you could also do with XML is crazy, and it only shows a very limited understanding of what XML actually is.

Transformations, schema checking, xpath queries, well defined linking and embedding, a host of tools that support applications of XML on every platform, the list goes on and on. JSON has none of this, it's basically exactly what its name says it is ('JavaScript object notation') and not much more. It covers maybe 10% of what XML and all its related tools and standards are. The current trend seems to be that XML is 'old technology' and all the cool kids use JSON, and it bugs me. You can't possibly sincerely say JSON is 'just like XML but less verbose' unless you simply don't really know much about XML at all.

YAML, which is a superset of JSON, adds the potential to be just like XML but less verbose. It never went through the same kind of hype though, so the tooling is not nearly as pervasive.

I must be one of the few who thought that XSL was a great way to build visual interfaces to data, in browsers that support XSLT (this was an area where IE had an very early advantage over other browsers, not sure what the state of things is today). Sorting, filtering, drilling down... all trivially easy and declarative with XSLT, once you groked xpath. Yeah it was verbose, but otherwise great.

There are some similar approaches for JSON data now, I think... but none are standards-based AFAIK, meaning they are mostly single-vendor or project solutions.

XML can do the same trick with indirect objects as COS, using ID and IDREF type attributes. A number of years ago I was dealing with the archival and retrieval/display of enormous medical textbooks in XML, and I couldn't efficiently pull out arbitrary elements (chapters, sections, paragraphs, etc) because of the hierarchical nature of the XML document structure. I had to parse the whole thing to use an XPATH to get the element I needed, and that took too long. (My parser could handle 3MB/sec, and some of these books were over 100+MB.)

The solution I came up with was a program that transformed the documents by flattening them into a relatively small hierarchical structure that represented the volume/chapter/section headings of the book, and a flat list of elements that were small enough to parse quickly. I inserted ID and IDREF elements to link these two parts together, and created an external index of the file offsets and lengths of each element in the flattened list. That let me use simple file I/O to access any element by ID, pull it out of the larger file, and only then start the parsing engine.

It was like the article mentions: my XML file, together with the external index (in a simple Unix DBFile file) was a miniature NoSQL database of the textbook.

BTW, this predated the "NoSQL" label, and was developed after testing of Oracle and the XML databases of the day completely failed to meet performance and scalability requirements. My solution has an infinite capacity to scale; its performance is not impacted by the number of books in the system nor their size. The retrieval and display time of a single chapter or any subelement is a constant proportional only to the size of that chapter, and is not affected by the overall size of the collection. All of the other solutions we looked at got slower as the number of books increased and as the size of the books increased. (I mention this only to head-off any comments about reinventing the wheel.)

ID/IDREF isn't exactly the same as indirect objects in COS. When you're parsing an XML document, ID/IDREF are just attributes like any other that get added to the DOM, and then application code can dereference them later if it wants. In COS, indirect objects are part of the serialisation format and the parser needs to understand them and dereference them in order to be able to parse the file.

For example, the COS "stream" object type is serialised as a settings dictionary, followed by the 'stream' keyword, the stream data, and the 'endstream' keyword. But what happens when the stream happens to contain the bytes 'endstream'? Well, the settings dictionary has a "Length" key that tells you how long the stream is, without you having to scan for the 'endstream' keyword. However, because most streams are compressed and compression makes it difficult to guess in advance exactly what the compressed size willl be, COS allows you to make the Length key an indirect reference to an integer defined later in the file. Like so:

    <<
    % Here, the value associated with the Length key is
    % indirect object 42 revision 0.
    /Length 42 0 R
    /Filter /FlateDecode
    >>
    stream
    ...compressed data goes here...
    endstream
    % And now we define object 42 revision 0
    42 0 obj
    12345
    endobj

So, the parser needs to know the length of the compressed stream in order to parse it, but it needs to have parsed the compressed stream in order to get to the length data. The way out of this catch-22 is the object index at the end of the file, which gives you the index and location of each indirect object. A COS parser needs to start at the end of the file and load all the indirect objects, cache them, then go back to the beginning and stitch them into the deserialised object graph as they're referenced.

How is COS separate from PostScript? It looks like a straight PS dictionary.

Well, PostScript probably doesn't have the indirect-object-reference index at the end, but I don't really know much about PostScript so I couldn't say for sure.

I wonder if PostScript is to COS as JavaScript is to JSON.

And here I thought everyone hated XML...

I don't hate XML, I just think it tends to be misused.

I (still) think XML is fine for configuration files and things like markup of UI/UX components (See: Flex, WPF, Android layouts, etc). In this context, where you may be modelling the relationship between many parents with many children nested fairly deeply, I find XML much more readable and easy to edit and reason about than JSON.

Where XML makes little sense is as an over-the-wire network protocol format or in any context where its "human readable"/"human editable" properties aren't as important. In those contexts, I think JSON is far better suited though neither is completely ideal.

I'm not keen on XML for configuration files, it's just too full of junk to read quickly.

I don't think XML is good at all for config files or UI/UX component markup, but for me that's mostly because of the end tags. Other formats, such as YAML, are able to describe hierarchical data without the verbosity of end tags, and I think that makes the files much more readable.

In fact, I think it's fair to say that WPF and all of Microsoft's config and resource file formats inherit their use of XML from mimicry of the Java ecosystem when C# and .NET were being developed as an alternative stack to Java. Android's inheritance is more direct, since it's based on the Java ecosystem. And as we all know, the crazy hype phase that XML went through was hand-in-hand with the crazy hype phase that Java went through. If Java hadn't happened, XML probably still would have as an improvement over SGML, but it probably would have remained the simple document markup language that it's pretty good for rather than becoming the "XML Everywhere" beast.

http://linuxmafia.com/faq/Devtools/parable-of-the-languages....

I love how some technologies cause some folks to get hostile ... HTML and XML are very close cousins in the same markup family.

On the basis of correct usage (e.g. not insane application of XML to corner cases) I don't fully understand the logic of people who profess love for HTML and hatred of XML, they are in the same family. The only difference is that HTML (now)is a controlled vocabulary, eventually folks will want to add markup of their own design w/o committee discussing the merits.

Perhaps next decade ;)

Reply on news.ycombinator.com