Hacker News Reader

Comment by creshal | Hacker News Reader

Reinventing XML, one data type at a time.

My thoughts exactly. JSON is great for Javascript clients, but if you're dealing with clients written in multiple languages then there is already a good language-neutral serialisation format: XML. Just because it's not fashionable (with some) doesn't mean it doesn't work.

Edit: So why the downvotes? How about a conversation instead?

I used to agree, but I've lately come over to the JSON side, I think. It's easier to read by a human, and it doesn't have this weird "should it be a node or an attribute" thing. Single things are properties, many things are arrays.

And now with Schemas and editor support for them, I think it is an acceptable replacement personally.

Nodes for data, attributes for metadata.

I think you made a good argument, although I personally prefer the more mature XML tooling and metadata support for versioning.

That's fine in theory, but it's not a universal heuristic, and even then, different smart well-intentioned people independently applying it won't necessarily come to the same result.

It's hardly answers the question, as strictly speaking there's no such thing as "metadata". Everything you can pass to another person can be described as "data", whether it is a color of your coat or the statement that you are describing the color of your coat. Basically, you can think of any attribute of an object as "metadata" as long as it can be only one per object and doesn't have any attributes itself, and both things can change easily and depend on the point of view.

On the contrary, I don't remember one single case when the object was described easily using "node/attribute" separation, but couldn't be described as easily using JSON. In fact, I don't even think it's possible, as you can always make two children for each object: "attributes" and "nodes".

So I guess it actually is unnecessary complication and not the benefit of using XML over JSON.

> Nodes for data, attributes for metadata.

Its a nice soundbite, but it ends up being less than useful in practice, because all metadata is data, and almost any data can be viewed as metadata, a distinction which is both subjective and strongly influenced by the use to which a consumer is putting the data rather than being determined on the basis solely of the inherent nature of the data.

Its a rule of thumb (heuristic). Judgement is still required.

(A soundbite is something altogether different.)

Also whitespace handling. XML is still based on SGML and its idea that a document is basically a single string of text with certain substrings "marked up" with metadata. This assumption is of course completely false for the overwhelming majority of XML use cases. Yet, it still influences many design decisions and processing steps in the XML toolchain, the most prominent being that simply formatting an XML document may change its contents.

JSON is a lot more "honest" in this respect, in that it's core data model is already useful for many applications, even without additional standards bolted on top of it. (Though those exist, this article being one of them)

Explicit arrays are what I consider the most important advantage of JSON over XML. In XML, every element is repeatable by default and only out-of-band specification can restrict cardinality. Two bytes extra per array, that convey so much structural self-documentation that is just not there in an XML document.

So JSON succeeded because XML is not verbose enough? Really did not see that one coming.

JSON is pretty great to parse in a variety of (web-relevant) languages.

And it's a good data exchange format if you're okay with its loose typing.

But if I need a strongly-typed, extensible markup language, I'd think really hard about inventing my own…

Sure, XML may be decent as an extensible markup language, but what if you need a strongly-typed data exchange format?

Turtle!

Well, I can dream..

I'm going to risk sounding thick here, but why would you need a strongly-typed data exchange format? I always thought the beauty of JSON was that it forced the sender to organize the data in a generic way, which allowed the receiver to interpret it however needed.

Without a strongly-typed data exchange format, you constantly have to write code that makes assumptions you hope the sender followed. Timestamps are ISO formatted. No, timestamps are seconds since 1970. No, they're floats including milliseconds. Blobs are base64. No, hex. Money is a string. No, it's a float. No, it's an int in micros. An empty array is different than an omitted field. No, they're the same. No, multiple values are comma-delimited strings.

> Money is a string. No, it's a float. No, it's an int in micros

No, it's a decimal number with attribute-dependant accuracy settings, with at least three different accuracy levels used in the same object used for different purposes.

(Just something I came across today at work. Thankfully, my SOAP library handles that automatically…)

But no matter what format any bit of data is in, it should still be documented. Then any consumers of the data are going to have to convert it into a format that suits their language and application.

> But no matter what format any bit of data is in, it should still be documented.

That's what types are: enforced documentation.

That's what types are: enforced documentation.

That's an interesting way to think about it, but I'm not sure it applies here. From the example given about money being a string or a float. A type would just enforce either or, but the consumer of the data would still need to know what it is, and convert it to whatever is necessary for them.

> what if you need a strongly-typed data exchange format

I'm not convinced that "strongly-typed" is an attribute that can meaningfully be possessed by a data exchange format. "Strongly-typed" is about allowed actions in processing data, not about the inherently-static format of data exchange or serialization.

http://programmers.stackexchange.com/questions/61198/if-xml-...

http://blog.codinghorror.com/xml-the-angle-bracket-tax/

http://nothing-more.blogspot.co.uk/2004/10/where-xml-goes-as...

SOAP's serialisation of RPC predated the popularity of JSON, and has largely been replaced by JSON for web-RPC through REST APIs. Why? Because it's needlessly verbose and complicated.

(I do worry that new serialisation formats are being developed in a vacuum, and we'll reinvent ASN1 or something)

The downvotes are because it comes across as a shallow, middlebrow dismissal.

An interesting and useful criticism would first engage with the strongest arguments of Rich Hickey (creator of Transit and edn). If you find something in his Language of the System talk (https://www.youtube.com/watch?v=ROor6_NGIWU) that you either (a) disagree with (b) think XML solves already, I and many others would certainly be interested in having that conversation.

Note that this doesn't mean the exposition of "Why Transit" can't be better, but that calls for constructive criticism on how explain the ideas better, or a question made in qood faith. What it doesn't call for is a hostile reply saying in effect "pff, already been done already, stop reinventing the wheel".

Transcript of that talk: https://github.com/matthiasn/talk-transcripts/blob/master/Hi...

XML is eXtensible (via DTD or XS), but JSON is a better Object Notation than any Markup Language.

Every XML parser I've seen is garbage on the performance side of things.

Personally I'm a fan of YAML for both being fast and human readable.

Hopefully something better than XML comes of it before the enterprise gets its grubby hands on it!

Enterprise here. We like XML because it's explicit.

   { number: 1.0000000000000000000000000000001 }

Try parsing the above JSON consistently in several languages without a consistent schema definition.

Feel free to write a schema definition for json data (or use the existing). Why should this not be possible?

Well in this case, JSON schema doesn't specify any more fundamental types other than "number". On the receiving end of a wire or network contract, how do we pick a storage type for "number"? We can't because the constraints of the type are undefined. Ergo JSON schema isn't a strong schema language. Be explicit is really important when defining contracts.

Type this in your address bar for an illustration:

    javascript:alert(1.0000000000000000000000000000001);

So where can we go here. Yep:

   { number: "1.0000000000000000000000000000001" }

which means we then break the encapsulation boundary of the metadata. Then we have a wire contract that says "this is a string" and a separate semantic contract that says "this is a decimal".

XML:

   <number>1.0000000000000000000000000000001</number>

Schema:

   <xs:element name="number" type="xs:decimal"/>

This is just one example. We can also serialize and deserialize complex self-relational composite types transparently at both ends of the channel.

This is a real world problem we encounter in the financial sector every day.

On Firefox, the maximum number of digits that yield values different to 1 are:

javascript:alert(1.000000000000001);

Otoh, what's the difference between

    { "number": "1.0000000000000000000000000000001" }

along with

    { "number": { "type": "ModelReal", precision: 64 } }

versus

    <number>1.0000000000000000000000000000001</number>

along with

    <xs:element name="number" type="xs:decimal"/>

In each case you have text data given meaning by an external semantics enforced via a schema. The real and unavoidable downside is that JSON actually contains a really lousy primitive. It wouldn't be bad except practically every implementation of JSON automatically performs a lossy coercion to IEEE floats.

Well, the `xs:decimal` example is quite interesting. In the XML Schema specification on xs:decimal (http://www.w3.org/TR/xmlschema-2/#decimal), you'll see the following note:

All minimally conforming processors must support decimal numbers with a minimum of 18 decimal digits (i.e., with a totalDigits of 18).

You have 32 decimal digits in 1.0000000000000000000000000000001, so "minimally conforming processors" are, according to spec, actually free to drop everying after 17th place after comma in this case.

You say "constraints of the type are undefined". I don't see how "double-precision 64-bit format IEEE 754 value" from ECMAScript spec is lesser defined than "decimal numbers with a minimum of 18 decimal digits".

JSON doesn't define the number format or precision which is a major issue. It's not IEEE754, its a string of digits. What you have done is prove the point that it's implementation specific, in this case JavaScript.

You're right about decimal precision however so I conceded there but the precision and capability is defined.

I just see no big difference between xs:decimal and JSONNumber. None of them is defined precisely enough to guarantee unambiguous handling of numbers like 1.0000000000000000000000000000001.

Then use this instead:

{ number: "1.0000000000000000000000000000001" }

How is it worse than XML?

See: https://news.ycombinator.com/item?id=10385610

Ok, now I see that you are complaining that the JSON Schema language is not strong enough. In this case I think people should create better new schame languages for JSON, as I find that the JSON syntax is better suited to data than XML, which is better suited to marked up text. (maps, arrays and atomic literals are fundamental data concepts, while tags, attributes, free text are closer to mark-up. )

The thing with XML is it's actually much lower level. There are no types inferred by an XML document. It's literally just chunks of data. JSON defines strings, boolean, maps, arrays, numbers.

It's conceptually easier to think about JSON with simple data sets but it's terribly inflexible and you have to think about how things are represented inside strings. A couple of thought exercises on JSON:

1. How do you represent an image inside JSON?

2. How do you represent a reference to another part of the data in JSON (consider a DAG for example)?

3. How do you represent an ordered set or an unordered set in JSON?

4. How do you represent an unsigned value in JSON?

As soon as you are making a schema for JSON you've eliminated XML with no good justification. More and more, people are re-inventing XML on top of JSON: schemas, namespace, the works. JSON is great because when you don't need all the things that XML solves it's extremely succint and readable. Use any single copied feature from XML and it turns out being more verbose and less readable than XML.

Use the right tool for the job.

"As soon as you are making a schema for JSON you've eliminated XML with no good justification"

Sure, if you don't consider "human-readable" a good justification. Some of us do.

People moan about XML readability somehow forget that it uses approximately the same syntax as HTML. Imagine HTML in JSON. Nope...

> People moan about XML readability somehow forget that it uses approximately the same syntax as HTML

No, they don't.

> Imagine HTML in JSON.

Yes, sure, JSON is a crappy text markup language, and would be much less readable than HTML for that purpose.

OTOH, readability when used as a markup language for content consisting largely of prose text and readability when used as a structured serialization format for data that doesn't mostly consist of large blocks of annotated prose isn't necessarily the same thing.

The point is, XML is quite good for unstructured, semi-structured and strongly-structured data and is more versatile from this point of view.

> The point is, XML is quite good for unstructured, semi-structured and strongly-structured data

That's a highly-subjective and controversial point.

(To me, XML is the Java of data languages -- its a lot worse than the best alternative considered on its own for almost any purpose -- though the best alternative will vary by purpose -- but it has a fairly wide range of uses for which its not intolerably bad, and its often a better choice than its inherent features would suggest because of the strength and maturity of the ecosystem around it.)

> best alternative

For 100% of the XML feature-set I don't actually know of a viable alternative. If you are using XML for the right reasons and the right way (rare) there is currently little or nothing that can replace. That being said, there are a vanishingly small amount of problems that actually require XML - namespaces and extensibility are two of them.

> For 100% of the XML feature-set I don't actually know of a viable alternative.

Real problems rarely need 100% of the XML feature set to solve. The breadth of the feature set is why there are lots of problems for which XML is a tolerable solution based on its inherent features (which in turn is a big factor in why it has such a large ecosystem), but they often don't make it the best solution (especially before considering the ecosystem, which is important in choosing a tool, but not a reason to avoid developing a new alternative, since any new alternative is going to start with an ecosystem disadvantage, but with adequate inherent value should be able over time to gather an ecosystem of tools.)

I should have probably said "equally good".

Try creating that number is several languages.

#/usr/bin/python n = 1.0000000000000000000000000000001 print n # 1.0

//Javascript n = 1.0000000000000000000000000000001; console.log(n); // 1

  from decimal import Decimal
  n = Decimal("1.0000000000000000000000000000001")

just because float is the default, doesn't mean you have to use it. JSON doesn't have anything else, though.

JSON does not have float (nor double) either. JSON has JSONNumber.

It's not valid JSON to start with.

I know. This was intentional and I'm glad someone noticed it finally.

Run it through JSON lint ( http://jsonlint.com/ ), then fix the error (unquoted number text), then run it again and watch the data loss occur due to my original point...

True, any problem domain that requires accuracy to 31 decimal places might not be best suited to JSON. I wonder how many decimal places would be required to represent the percentage of projects that would actually be affected by that...

That's not the problem I'm outlining. It is that semantics around type handling are implied rather than specified by schema.

I don't understand your point. The semantics around type-handling are very clearly defined in the JSON spec: a number is simply a series of digits (plus decimal point, - sign, etc.). If you choose to interpret that JSON in a language with less than infinite precision, then you need to accept that some data loss can occur. That will happen if you're mushing an XML string into a number type, too.

I'm also not sure I'm getting you. In JSON, types can be unambiguously deferred by syntax/via gramar. You can't mistake JSONString for JSONNumber/JSONNullLiteral/JSONBooleanLiteral, the type is absolutely clear from the syntax. So I don't see how a schema would help here. What would the schema express? Surely not "this is a number" as it is clear without the schema. What then?

And what exactly do you mean by "semantics around type handling are implied", especially in `{ number: 1.0000000000000000000000000000001 }` case?

If Transit is just a human readable XML that is easy to use, that's a great thing because people will actually use it!

Not really. Transit doesn't care about the underlying data-layer. It could be JSON, could be msgpack, could be XML. Currently, only JSON and msgpack backends exist. XML is probably ruled out due to performance.

Why the downvote? XML could also benefit from automated tag-shortening (in my last place of employement, we encoded XML-tags and attribute-keys manually).

Reply on news.ycombinator.com