JSONx is an IBM standard format to represent JSON as XML | Hacker News Reader

JSONx is an IBM standard format to represent JSON as XML

publib.boulder.ibm.com

52 pointsrpeden75 comments

"In an XML Firewall service, JSONx can be used like other XML input."

http://publib.boulder.ibm.com/infocenter/wsdatap/v3r8m1/inde...

"An XML firewall is a specialized device used to protect applications exposed through XML based interfaces like WSDL and REST and scan XML traffic coming in and out of an organization. [snipped] XML Firewall is often used to validate XML traffic, control access to XML based resources, filter XML content and rate limit requests to back-end applications exposed through XML based interfaces."

http://en.wikipedia.org/wiki/XML_firewall

Also, http://en.wikipedia.org/wiki/XML_appliance and http://www.layer7tech.com/products/xml-accelerator

Astounding, is it not?

It's less silly than it sounds. Many XML attacks have JSON equivalents. Do you know how your JSON parser will deal with 512KB of open square brackets in a row?

You've (both) opened my eyes to a new and terrifying world. I don't know if I should thank you or just weep silently.

all common XML message formats including POX (Plain Old XML), SOAP, REST and AJAX

That made my day.

My brow is now permanently furrowed and my heart has arrested.

I am sad.

Wow. Just wow.

Probably controversial, but in a dozen or so years of programming I think XML probably represents the most stupid and costly ideas to've gained widespread adoption.

I honestly can't think of a single other relatively recent technology that's responsible for nearly as many wasted lines of code, unbelievably inefficient systems, and legions of developers who are terrified of bytes that don't render nicely in notepad.

I hate it. Just saying. (Downvotes expected).

Agree completely if you're talking about XML used for storing structured data, as a serialization format, or as a settings format.

Disagree completely if you're talking about XML as an extensible markup language for textual documents, for which it is a perfectly sane choice. Problem is, few people use it that way.

Couldn't agree more.

The thing that frustrates me the most is that people jump to defend it because it's what they know. It's like they're plugged into the matrix and they will fight for it until the end.

People love to write regexes to process their bastardized XML subset because it's "easy," without realizing that their regex is not even remotely an XML parser and would fall over at the first sight of CDATA.

Totally. On two totally independent occasions I've come into work to have a coworker tell me they implemented an XML Parser the evening before. Um, right. Great.

The thing that totally boggles my mind is that I think it really prove-ably fails to deliver on literally every promise/benefit/design-goal that I hear folks claim it embodies.

I know parsing XML with regular expressions is a sin, but when you need to get some data out a 20 GB XML immediately in the next 10 minutes. It turns out that its better to check out the layout of the XML, use combinations of grep, sed, awk , perl with pipes just to serve the need for the moment.

This is better than spending next 5 hours writing a Java program to do it. Its not a permanent solution, but either way that Java program and the shell hack are both going to do the very same.

I would rather use the shell and do it, rather than spend the next 5 hours with Java and eclipse.

I've similarly seen simple pattern matching used in high transaction volume processing. Using a DOM based, or even a stream based processor is considered too expensive. So, just look for the relevant tag and grab the next bit, while presuming that the context is correct. (This involved a limited number of trusted partners, in a relatively steady-state environment.)

OTOH, it eliminated discussions/arguments about what format to use. In a corporate environment, that can be a significant win in and of itself. (Yes, "win" should be in quotes.)

The fact that those are your two options is just a testament to how absurdly unsuitable XML is for this purpose.

Your "combinations of grep, sed, awk, perl" will corrupt your data if your file happens to contain any CDATA, entities, unexpected formatting, processing instructions, etc.

Don't get me wrong, sometimes a quick hack is worth the risk. But it's certainly no reason to defend XML, which is what put you in the position where a quick hack was your best option.

I agree with you. Actually I'm not too much in favor of XML either. Somebody high up in the hierarchy attends a few conferences here and there, reads the IBM journal. All along the word 'XML' gets repeated some 1000 times.

So the impression he gets is, XML is going to add value to everything it touches. Most developers like me have no say in that at all, we have to just use it.

I prefer the hack to get it just done. But agree with you, none of that is a permanent solution.

> I honestly can't think of a single other relatively recent technology that's responsible for nearly as many wasted lines of code

Java.

Is Java verbose? Totally.. Is it meritless (like XML)? Absolutely not. I think Java's a great language for some problems, and a fine language for loads of problems.

It's a shame that it's overused for what I guess I'd only describe as institutionalized laziness.. But I don't think it deserves to be mentioned in the same conversation as XML (when the topic is Technologies That Are Fucking Awful).

I agree. I was unfair. Java is good for many things - many times you want to control details somewhere between C and, say, Perl. And yes - it's been used for all the wrong things.

There is one good thing XML. brought: BPEL.

It's the funniest language I've ever seen.

Reminds me of the Property List evolution:

1. "Under NeXTSTEP, property lists were designed to be human-readable and edited by hand, serialized to ASCII in a syntax somewhat like a programming language."

2. "In Mac OS X 10.0, the NeXTSTEP format was deprecated, and a new XML format was introduced."

3. "Since XML files, however, are not the most space-efficient means of storage, Mac OS X 10.2 introduced a new format where property list files are stored as binary files."

Everything converges to the equivalent of protocol buffers.

Property lists -> XML -> Binary Property Lists

JSON -> JSON Schema -> Binary JSON

XML -> XML Schema -> XML Namespaces -> Binary XML

Or you can skip all that faffing around and just jump straight to protocol buffers. :)

ps. Protocol Buffers have a text format too that looks very similar to JSON. And you can encode protocol buffers as JSON, though I'm not sure there's good software support for it yet. I'm working on it.

Similarly you could use Doug Cutting's Avro format that does have a JSON representation. It is also evolvable like Protocol Buffers and works well in untyped languages. I highly recommend it.

Avro looks like an interesting format, thanks for the pointer.

Unfortunately, XML still prevails in XCode config and xib files. Modifying them makes for a completely polluted diff in your commit. Makes me rage.

The objective of jsonx is: 1) a loss-less representation of json in xml so it can be reused with xml tools. In a sense it's the reverse of jsonml.org.

jsonx is not (neither it is meant to be): - a replacement for json - a format that enables you to query json in a xml database - a standard (I would remember if I was part of a standard comity, although I do recall playing bullshit bingo in some of these meetings)

I think the use case for jsonx is extremely small: maybe exclusively related to transmitting json over the wire in hardware that is specialized for xml processing. Having worked in it I argued many times that it is not even a queryable format, which makes it less usable even in "xml world". ( for a queryable format example you can check isubiker/mljson@github )

If you are not trying to achieve 1) you can/should simply disregard jsonx. That's certainly what I do: It's completely irrelevant for me and even thought I worked on it I never used it in any of my projects. Why should I? You can't query that crap for the heck of it.

About saying crap about IBM: That's just plain stupid. The fact that you can't understand something simply means you can't understand it and not that it is of no value. "all I know is that I know nothing." - Socrates, supposed to be smart guy

This whole thing about not speaking about something wrong ultimately leads to right stuff being lost. Enterprise vendors like IBM are always in the hunt of giving the industry something that forces volumes of people to use it, and then design tools around it that make people addicted to it.

Just put a auto completion feature in a couple famous IDE's to enable this happen, and see how many people use it without caring what, why and how of it.

Then one fine day there is situation that most of the programmers in the market are trained to use it and if you have to run a project you to hire them and let them to what they want. And then the technology adoption grows.

Its not OK just to remain silent. Proper criticism is needed often, it helps things evolve in the right direction.

Not only is XML shoe horned for purposes where it doesn't belong to, but also people who know only XML tend to use it so badly that it brings bad name to technologies associated with it.

I work for a project that has involvement in a lot of technologies like Java, Perl, C and C++. Now the problem is somehow there is assumption that its difficult and costly to hire good C/C++ programmers. So what is the path taken? They tend to replace everything with Java. Now comes the actual issue, its easy to hire programmers who know Java. They come in volumes. Just post a hiring Ad, and thousands will land up at your doorstep next day. The actual issue is how do you separate the good ones from the bad ones?

Once you hire such programmers, you have to deal with technology addiction. Things like eclipse, XML etc the whole project begins to revolve around such things. In my project nearly everything is XML, config files, DB, persistence you name it. If its on a file its XML. This JSON in XML which is surprising most of you folks already exists in a lot enterprise code bases. Project managers seem to have a narrow vision, debates on issues like these terminate with 'It works, so we just don't care'.

I'm not against IDE's. But when your programmers can't do a simple deployment on test boxes, can't figure out even simple commands. All under the excuse of 'I only use eclipse', It begins to show up on the language community. Java in it self is not bad. But by reducing the barrier to entry to such low levels you are opening door for all sorts of toxins to come in.

I see Python is the next Java in making. Masses are flocking to it. As it usually turns out to be, most of them horrible. They just contagiously corrupt the whole community, spread the disease and once they are done destroying it they move onto some thing new.

After all you need to do something for a Job.

Is it April 1st again already?

This comes from the docs for the DataPower XML Security Appliance. Naturally, I was curious about what this Appliance does.

If you go up the tree a bit, click Development, then XML Firewall, then Introduction, you'll find this tidbit:

"These appliances offer an innovative, pragmatic approach to harness the power of SOA while simultaneously enabling you to leverage the value of your existing application, security, and networking infrastructure investments."

Seems like a great attempt to use as many words as possible to say nothing at all. :)

This point seems relevant:

"Bridges to Web 2.0 technologies with JSON filtering and validation, support for REST verbs, and converting/bridging of REST and Web services."

http://www-01.ibm.com/software/integration/datapower/xs40/

There's a service that's being subscribed to that provides video that I can't share, sadly. However, when I reverse engineered how our video player was translating a uid into a link, I was horrified.

Essentially, we perform a request that returns an xml document wrapped in a json object, wrapped in a jsonp callback, that is then parsed by a php script on our end back into a pure json object.

This made me die a bit.

Why oh why... If it helps JSON invade the enterprise then ok, but why make a good thing bad.

http://publib.boulder.ibm.com/infocenter/wsdatap/v3r8m1/inde...

To allow you to use XSLT tools apparently.

XSLT and XPath is about the only reason I can see to use this. But, you know, how hard is:

   x.person.addresses[0].city?

I guess XPath is easier when you probably have to do something like this:

   JSONObject json = (JSONObject) JSONSerializer.toJSON( jsonTxt )
   String city = json.getJSONObject("person").getJSONArray("addresses").getJSONObject(0).getJSONObject("city");

Or however you'd do it in Java.

That's not hard, what's hard is

   //person[@alive="true"]/pets//dog[@hair="long"]/*/flea[@name]

still, i think an xpath evaluator for json would have been the way to go here instead

You appear to be confusing Java and Javascript.

JSON stands for Javascript Object Notation.

In Java the equivalent would simply be serialization.

NB: if XSLT and XPath is the solution, you've already lost.

JSON is used for far far more than javascript these days. It's a far better communication protocol than XML for most things.

"Some people, when confronted with a problem, think 'I know, I'll XSLT.' Now they have two problems."

(edited for relevance)

This is actually one of the few things XML is very good at: embedding a document within another document.

I mean, it's not something you'd want to use all over the place, but there are situations where it might come in handy.

wow, the brevity and parsability of xml combined with the specificity and structure of json.

Not sure what all the fuss is about. It's actually neat to have a common way of changing JSON to XML, think of a web service that offers an API using our beloved JSON. But wait, we also want to provide an XML API! Let's design our own, ad-hoc, crappy mapping or... use that thing that everyone else is using.

XML is out there, has lots of tools and libraries and is actually useful for some things. Not to mention the array of technologies around it, like XPath, XQuery etc, that JSON is lacking (probably because it doesn't need it, because these formats do address slightly different requirements. Gasp!).

"the fuss" is because a lot of people jumped on the JSON bandwagon in order to get away from XML.

That's fine for them. My question stands: what's wrong with someone proposing a generic way of transforming JSON to XML?

Then wouldn't it make sense to allow people who are stuck using XML in some places to convert to JSON when talking to those that aren't stuck with XML?

> But wait, we also want to provide an XML API!

Why would you want to do that? Govt contracts? Purifying your soul through suffering?

All I want from my XML APIs is to give error messages telling the user to upgrade to the JSON ones. ;-)

> XML is out there, has lots of tools and libraries and is actually useful for some things.

A lot of things had to be invented in order to make XML useful.

Not to go against the grain, but this could be useful. Say you already have some sort of datastore that uses xml. Now somebody hands you JSON for some reason, maybe a list of companies. Convert it to XML, stick it in your datastore and use XSL to do whatever you want with it.

XSLT is a scary language. It took several years for computer scientists to prove that it's Turing complete.

IBM. I know you have smart people. Why don't you let them be smart?

IBM has been divesting themselves of good programmers for a while now, probably at least a decade.

Their interns are steered away from programming, towards career paths that are perceived as adding value: e.g. consulting.

I suppose in theory this makes sense: have the smart locals figure out what is needed, write up some specs and then get the guys in Bangalore to bang (sic) it out for you.

In practice, it is an absolute dogs bollocks.

(1) The local IBMers have forgotten how to do anything in the real world. No, seriously. They can't even install a server to use for their own use as a test machine in less than 4-6 weeks.

(2) The Bangalore IBMers cannot code for shit. They are the second worst programmers I ever worked with (and the worst was a consultant Thoughtworker who was actively sabotaging the project). The ones who did the least got promoted. twitch

(3) They can't even maintain the software they wrote 5-10 years ago, they just don't make any effort to retain that knowledge.

(4) It might be different for US IBMers, but outside of the US the programmers got the shaft big time when the GFC hit. Stupid little things like (even if your business unit was profitable) you had to bring your own coffee and milk in to work because they stopped supplying them free.

Other than the above, IBM is just like any other large organization.

The thing about Bangalore IBM'ers I'm from Bangalore myself and can say that you are 100% correct. The whole thing is IBM Bangalore is actually composed of IBM services, and most of them service projects. I have many friends working there, and its just another Indian company. Its your usual job hoppers who know nothing but keep moving from one company to another just for hikes.

To more about the state of affairs here, read my comment on another thread - http://news.ycombinator.com/item?id=2480368

> They can't even maintain the software they wrote 5-10 years ago, they just don't make any effort to retain that knowledge.

IBM is the company that keeps a nearly-five-decade-old architecture and ISA around for its business customers, correct? (zSeries (or whatever its name is this week) is derived from s390, is derived from s370, is derived from s360, was created in the early 1960s.) z/OS and z/VM are modern versions of MVS and CP-40, which date to 1974 and 1967, respectively.

It seems odd that the company that does that would have problems with decade-old software in any of its business units.

It seems Stormbringer was looking at the IT consulting division. I have to agree with him/her on that - I hear horrible things about them and I'd never, ever, under no circumstances, hire them.

That said, I have enormous respect for their mainframe division and operating system programmers. Their server hardware (z, i, and x), z/OS/, z/VM, AIX and their contributions to the Linux kernel are all very good.

IBM is certainly big enough and old enough that different divisions can behave like completely different companies. It's a bit unfair, in fact, to see one division tarnishing the image of a company so full with groundbreaking accomplishments and such a rich history as IBM. The consulting division has to compete for huge contracts, that are managed by pointy-haired lifeforms who can't see beyond hourly rates, with equally incompetent organizations like EDS/HP, Accenture and Tata.

You put an evolutionary pression on the market driving it to provide low prices. The marker responds optimizing for price and price alone.

When you have to compete in price with incompetent-but-cheap organizations, you either become incompetent-but-cheap or get out of business.

Tough choice.

Big companies divide their company internally into many units. Things like research, services or even on basis of Industry verticals like Banking, Communication and Entertainment.

The units in IBM that work on Mainframes and Supercomputers et al, are totally different than the ones that execute maintenance and service projects. The people, the culture everything is different.

IBM has done a great job of building an organization that protects itself from bad employees -- which, at that scale, you're BOUND to have. The problem that I noticed in my time there is that this is done at the cost of not letting the smart people be smart.

Mmmm.... no. IBM has built an organization that protects the bad employees. Business Analysts that cannot analyse anything to save themselves for instance. Coders in Bangalore that cannot code and have shitty communication skills (but marvellously developed excuse making skills). Server admins that don't know how to install and set up servers.

I agree with the not letting smart people be smart... but it's not even that. If a project has caught fire and is burning down, they don't let the people with the firefighting skills come in and save it.

Partly it is an incentivization problem. That is, there isn't any incentive to be efficient and good at what you do. Especially if you're billing your people out as consultants by the hour and somebody else is paying for it. This is a two level problem. The individuals have no incentive to make themselves genuinely better at what they do (there are personal development programs in IBM which people are encouraged to take, but that just pushes them even faster towards positions where they don't actually do anything useful - e.g. management), and the organization as a whole actually loses money if it becomes more efficient.

India/Bangalore/Rest of India situation has become a joke to say at the least. Words like 'Architect', 'Consultant' , 'Tech lead' are all dangerous words here, generally a person who has that designation is useless, hasn't done any work at all.

During 90's and early 2000's most people got rapidly promoted regardless of whether they deserve it or not. They have hardly coded, spent most of the times at on site locations made good money and are now just relaxing and pushing time. Managers have no frigging clue how projects need to run, architects don't even have basic technical skills.

The assumption is that you can just give orders to juniors and get the job done. That is how most of the so called seniors work here. Good hackers/programmers are never rewarded due to many reasons. The horrible ones hop jobs to get good hikes and ultimately end up earning more than the good guys.

The general attitude of the guys joining the industry is not be technically good, never code, just get promoted quickly, go to onsite, earn money in USD, purchanse a flat, car, marry and settle down. And these sort of guys were actually rewarded during 90's and 2000's. So now everybody wants to be like that.

The actual good guys are there, but are lost in the crowd.

http://news.ycombinator.com/item?id=2442977

https://twitter.com/zedshaw/status/64050059276337152

Yo dawg, I heard you like serialization formats, so we put a serialization format inside your serialization format.

JSONx has a very limited use case, A quick check will show that the documentation points to its use in devices on the edge for transforming/bridging/proxying http://bit.ly/ky0q53 http://bit.ly/iDF2Y5

Could this present any utility as a communication interface between an RDBMS with XML support (DB2, MSSQL) and a document DB (MongoDB, CouchDB) ?

I know I'm supposed to be insightful in my comments on HN, but this is... I just can't.

Pretend I put the facepalm ASCII-art here.

I've up-voted this solely for the irony. Who says IBM doesn't have a sense of humour?

Why don't we yet have XMLj?

Because JSON supports only a subset of the features of XML.

Yeah, you know the subset that people actually use.

XSLT and XPath are tools that really feel like they should be useful, but for small things, I feel like a streaming parser in <favorite scripting language> is faster and more readable, and for large things, deserializing it all into <favorite database> and querying it that way is better.

I liked this better when it was called WDDX.

JSONx: In Space

I just threw up.

Using the example from http://publib.boulder.ibm.com/infocenter/wsdatap/v3r8m1/inde...

muk:~ fiam$ echo '{ "name":"John Smith"... }'|wc -c

303

muk:~ fiam$ echo '<?xml version="1.0" encoding="UTF-8"?> <json:object> … </json:object>' |wc -c

904

I don't really have anything else to say.

Edit: formatting

you've got to love the title of the linked page: "help"

there's someone in there, trapped...

The example may be unnecessarily verbose (for example, they would have saved 90 characters by using json as the default namespace), but this is text. It's ridiculously compressible. Saving a handful of characters here or there is probably wasted effort.

There is interesting side to this example of yours. Terseness and its power have never been appreciated by the masses. The average programmer needs a tool, that can match his level of competence. This isn't just true in Data formats but also true when it comes to technologies.

Java and Python, are loved by the masses today for that very reason. So as long as eclipse and its Ctrl-Space takes care of 'filling up the blanks' most programmers will use it. Beyond that everything is irrelevant at least for the masses.

Now one fine day a cool Perl/Lisp/C programmer comes around and shows a you how you could do all that with more compact code along with flexibility and power and people suddenly realize what fun they were missing for years.

Verbose stuff tends to get very famous for a while and then just fades away with time, never to be seen again.

Silly, but interesting. It just shows the power and simplicity of the JSON syntax to define these universal data structures.

Reply on news.ycombinator.com