Hacker News Reader

Comment by jwilliams | Hacker News Reader

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems).

I must admit I was scratching my head on this.

The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? There are valid non-spec concerns with order.

Replies here seem to assume stupidity here. It's a valid reason, but it's not the only one. Equally, the author doesn't ask "why" - why would it take so long, why was that option enabled?

View on HN

I can't help but think of the story of Chesterton's Fence [1].

>There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, 'I don't see the use of this; let us clear it away.' To which the more intelligent type of reformer will do well to answer: 'If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.'

[1]: https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fen...

If you require serialized JSON in a specific order, you are not requiring JSON, but some dialect of JSON and that should be clearly communicated. If, for some reason, a client needed a special serialization, i'd ask them for a spec, because the JSON spec does not apply anymore, and be done with it.

I think my argument is a bit different -- sure, this is an interface (maybe to a COBOL copybook in this case?). And yes, it's not "JSON".

Agree it's a leaky abstraction. Equally agree it should be documented. Often legacy systems have weirdness after seeing decades of edge-cases. Weirdness that makes them robust in all sorts of unlikely ways. Equally makes them poorly documented, leaky, opaque, and frustrating.

But I certainly have a lot of pause with the sentiment of "some idiot checked the JSON order checkbox on DataPower". My first thought is instead "I wonder why someone thought this was necessary."

Yes, that sentiment is highly unprofessional. 9x6 months translates to "that box is checked for reason" and "it's not going to change". Even if things are going to change, it's probably going to affect one function on your part and you are producing valid JSON anyway. Surely you can bill a fortune 500 company for a function to put things in order.

Money down 3 months later:

1) the gateway or more likely something behind it starts crashing for memory errors when someone is serializing some stupidly large array into it

2) using ordered vs non-ordered keys on input changes the behavior and fixes it

3) enabling this checkbox to prevent it fixes the issue and guards against it happening again

4) everyone has forgotten about this web weenies inquiry into the issue

5) due to #4, it takes like 3 weeks of system crashes and hairy debugging to find the cause

6) this guy is long gone, having ridden off into the sunset of smugness

Also:

1) fixing the serialization-dependent memory allocation 4 layers deep on the back-end to allow things to operate safely without this checkbox requires a complex change to several other components and associated system-wide validation testing which would take: drumroll "9 people 6 months to complete"

2) What is the point of this javadoc reference and how exactly does it relate to the issue at hand? Here's a random code doc reference too!: https://pymotw.com/3/collections/ordereddict.html and here also a discussion about impacts: https://mail.python.org/pipermail/python-dev/2016-September/...

Money down:

1) Nothing happened, this already went down years ago, as the author writes.

2) The setting was only stopping the BS IBM system layer, wasn't really needed elsewhere.

3) The BS setting was probably even default, e.g. not consciously enabled by someone for the specific system.

>What is the point of this javadoc reference and how exactly does it relate to the issue at hand?

The point is that the IBM system who had this setting was using that BS, brain-dead, not-JSON, implementation to handle the order, when the setting was enabled.

>I think my argument is a bit different -- sure, this is an interface (maybe to a COBOL copybook in this case?). And yes, it's not "JSON".

If it's not JSON, it's undefined when it uses JSON tools and libraries, and anything goes.

If it's not JSON, document it as your own dialect, and use your own tools, and try to never talk to the outside world based on it with systems that treat it as JSON.

>But I certainly have a lot of pause with the sentiment of "some idiot checked the JSON order checkbox on DataPower". My first thought is instead "I wonder why someone thought this was necessary."

It might not be, it could be the BS default setting...

> JWT needs it for example (and I'd assume many signature models)

You almost never need canonical representations for signing things. I would even say that if you need a canonical representation to sign your things, then that is a design smell of your cryptographic protocol.

You don't _need_ canonical representations for signing, but then you can't let go of the representation used for computing the signature. What is the argument against a requirement to only sign canonical data?

Could you please care to explain more? What is the "design smell", and what is the alternative solution to canonicalization?

I didn't mention canonicalization. My point is that serialized JSON is ordered - which I think is exactly the same property you're referring to.

That's an implementation detail. Serialized JSON can also be printed, but your systems shouldn't depend that JSON is always ink on paper.

>The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it

Whether the file has an order is an implementation detail. There might be no static file at all, for example, it could be an endpoint returning you the JSON, and it could return different orders for the exact same items every time you call it. That would still be perfectly valid.

If you "need it", don't put your data into an object, put it in a list of objects.

If the underlying layers need "ordered objects" (in other words, ordered hashmaps) they don't really use JSON.

This isn't really an issue of spec because these systems recognize a completely different protocol 'OrderedJSON' which interprets {} as an ordered list of key-value pairs instead of a set.

The mistake is assuming that two syntactically compatible protocols were actually the same protocol. uint =! int.

Only there's no such protocol. It's some adhoc implementation within a single system (or a single IBM ecosystem).

Justgoogled DataPower, out of curiosity, and I found XML Accelerator XA35. I did not know there is HARDWARE for XML PROCESSING! Blows my mind.

Yep, back in the SOAP days we used to have a lot of XML flying around that would need to be converted into one format or another via XSLT. You can certainly run XSLT on a piece of XML in your own code, but having a dedicated piece of hardware do it for you with a slick GUI was always an easy sell for IBM to management.

Could you elaborate? I am really curious how this hardware worked.

Was it a HTML Gui with file upload and everything? Or just some server, port and proprietary protocol?

Maybe these things are on Ebay for cheap. Oh, that looks like fun.

Impressed to learn about this hardware for XML processing. I came across an old article that mentioned the price started at $55K!

No idea why someone needs ordered key-value-pairs. I had the please of working with a device that relied upon ordered query string parameters. Why? Because they encrypted the values with RC4.

> JWT needs it

I already had at least four solid reasons not to use JWT. Still, adding another just keeps the dumpster fire burning.

By which I mean, in the most oblique and backhanded fashion, that there’s not going to be any reason for validating JSON order that doesn’t ultimately reveal or confirm a design gaffe somewhere down the rabbit hole.

Reply on news.ycombinator.com