XMPP For the Web with JSON | Hacker News Reader

XMPP For the Web with JSON

blog.superfeedr.com

44 pointsjulien28 comments

Hi all,

As the author of xmpp-ftw I feel I should throw some comments into this discussion.

Firstly, xmpp-ftw is not about starting an XML/JSON holy war, its about lowering the barrier to entry as far as possible. So see it as more of a gateway drug :) *

At present the tiny amount of effort it takes to build a proprietary (legacy) chat system with other technologies means that XMPP is somewhat being left behind (on the web) despite offering a huge number of benefits. Xmpp-ftw tries to make it as easy as possible by providing developers a format they are comfortable with and the ability to send/receive data with the minimum of fuss.

This project actually came out of discussions at XSF summits I've been attending. No-one (myself included) believes that XML should be replaced with JSON in XMPP, that would be a foolish notion. Generic JSON <-> XML is awful and doesn't quite work. The idea with this project was to used named events to fill in some of the missing information for building stanzas, and telling the user what is happening.

The view of JSON with XMPP has changed over the last couple of summits has changed, going from being referred to as the J word' through to open discussions about what can be done (the XSF seems like a very progressive group). No more jokes about XEP-0295 :)

If you want to see a better client library trying to do something similar check out Lance Stout's stanza.io https://github.com/legastero/stanza.io which came out of the same meeting.

So in summary: Xmpp-ftw is attempting to drop the barrier to entry for XMPP as far as possible and make it quick and convenient for developers to implement. Its not about replacing the angle brackets :)

Cheers, Lloyd

FYI, yes I've used socket.io, but you can stick anything that implements an event emitter and has callbacks and have it work. Currently I'm looking at wrapping sockJS.

* Drugs are bad m'kay!

I don’t understand the point of taking an existing protocol – with fairly well defined syntax/semantics, which has presumably been tested at various parts of the stack, likely has existing conformance tests, &c. – and wrapping it in a new ad-hoc protocol which has (some subset of roughly) the same semantics, just for a syntax change? Big addition of new complexity for little tangible benefit.

Why not just use one of the existing Javascript implementations of an XMPP client, or if they’re unsuitable for whatever reason, make a new one?

Lately there's been discussion within the XMPP community about how web developers tend to prefer semi-proprietary connectivity solutions like Socket.io even though the XMPP standard and surrounding solutions predate these kinds of things. We believe a large part of the problem is the encoding format.

For better or worse, the web (and I mean all of it, not just browser<->webserver side) is transitioning to JSON for just about everything. Yes, it often seems like pointless wheel reinvention, but that's what is happening. May as well embrace the trend in order for our work to live on.

One funny thing about all this is that when XMPP was first created (as "Jabber" in 1999), XML was selected as the format because it was the current trend. In that sense XMPP was super hip in the beginning. Hard to believe now people look at XML and think it's a relic. With that in mind, I wonder what will happen to all these trendy JSON protocols being made today? Dinosaurs of tomorrow, probably. My feeling is that it's the underlying concepts that will live on, regardless of format.

I don't want to start some kind of "XML vs JSON" holy war, but I really do prefer JSON over XML because of how compact and readable it is.

I really don't see JSON as a passing trend, I see it as (almost) the most compact, succint way to express hashmaps in both human-readable and machine-readable form. (XML is big, bloated, and bureaucratic.)

(It's tragic that Crockford left in all the qoutes around the keys, though. We really don't need to type those quotes.)

For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this.

> For the past couple of years, I really have been wishing for an "XMPP, but built on JSON, not XML". Really. I want this.

The X in XMPP stands for extensible not XML. The reason XML fits the bill is because it has concept of namespaces, what JSON has? If you don’t have explicit namespace support how do you make your JSON variant of XMPP forward‐compatible? You don’t without making it look much, much, worse than any of the examples I’m about to show below.

> (…) I really do prefer JSON over XML because of how compact and readable it is.

> XML is big, bloated, and bureaucratic.

My standard response to this is always “show me a JSON fragment that is shorter or prettier looking than an equivalent XML fragment.”

XML can certainly be shorter than JSON and often is, and repeated tags are the best showcase for it:

    <user id="abc">
       <phoneNo type="home">123456789</phoneNo>
       <phoneNo type="work">321654987</phoneNo>
    </user>

This turns into this beautiful JSON:

    [
      "users": [
	{
	  "id": "abc",
	  "phoneNos": [
	    { "type": "home", "value": "123456789" }, 
	    { "type": "work", "value": "321654987" }
	  ]
	}
      ]
    ]

Oh, but what if order is important? Take this example:

    <policy id="123">
        <drop someFilter="ghi" />
        <accept someFilter="abc" anotherFilter="xyz">
            <comment>
               This comment is quite long and spans several
               lines of text in pretty printed XML. 
               But this is quite alright, because XML
               by default strips extra whitespace.
            </comment>
            <!-- That’s another kind of comment, but this one is not meant
                 to be part of the data, unlike the one above. 
                 Some would say something as basic as comments is
                 an essential part of so called human‐readable format… --> 
        </accept>
        <drop someFilter="foo" anotherFilter="bar">
            <reason statusCode="690">No reason</reason>
        </drop>
    </policy>

Now you have to do it like that:

    [
      "policies": [
	{
	  "id": "123",
	  "rules": [
	    { "type": "drop", "someFilter": "ghi" },
	    { 
	      "type": "accept",
	      "someFilter": "abc",
	      "anotherFilter": "xyz",
	      "comment": "Unfortunately the best you can do with JSON is to rely on your editor’s dynamic wrapping. And this is how you break the line: \nWhat a treat!",
	    }, 
	    { "type": "drop", "someFilter": "foo", "anotherFilter": "bar", 
	      "reason": { "statusCode": 690, "message": "No reason"}
	    }
	  ]
	}
      ]
    ]

For an example of how horrible JSON in wild can really be see Reddit’s JSON API. Because JSON has no syntax for metadata you get things like that:

    [ 
      {
        "kind": "thing-foo",
        "items": [
           # 
           # list of comments or something that goes on for
           # two screens
           #
        ],
        "votes": 15
      }
    ]

See the keys are sorted for convenience.

XML’s text representation is very minimal and consistent. You don’t need to quote Every Single Thing, and there are no special cases for not quoting just some type of values. The only redundant part of XML is repeating the name of the closing tag if the </element> has value. But this actually aids readability, as demonstrated above.

Editing complex JSON by hand with a regular text editor is a nightmare, because of all those single '}', ']' and commas trailing for several lines when there’s any light nesting involved. Editing XML by hand on the other hand is no big deal.

Thank you for this, I have bookmarked it for future reference!

I've always made the second part of your argument: that namespaces are the key feature that make XML useful, especially for the eXtensibility in XMPP. I also think that namespaces are a part of XML that few people understand, partially because many XML/XMPP implementations don't even (fully/correctly) support them.

The rest of what you say is new though! The solid examples you gave for compactness/ordering were impressively concise.

I'd even go further and say they generally illustrate the difficult readability that can arise in JSON with not-so-difficult requirements--I had more trouble following the JSON than the XML here, by far. I'd say the embedded comments and possibility of a schema in XML are also a bonus, to help understand the meaning if it isn't obvious.

I am by no means an XML-supremicist, but I do take the unpopular view that it is a much more future-proof and natural solution for many more use-cases than it gets credit for. I'd love to see a successor that cuts out some of the warts as much as the next person, but every solution has pros/cons, and XML is a pretty good balance in a lot of cases where people refuse to look at it.

------- Quoting the parent:

This turns into this beautiful JSON:

    [
      "users": [
	{
	  "id": "abc",
	  "phoneNos": [
	    { "type": "home", "value": "123456789" }, 
	    { "type": "work", "value": "321654987" }
	  ]
	}
      ]
    ]

------ ... endquote.

That's a strange, contrived example. In Javascript, I would write it like this:

user = {id: "abc", homePhone: "123456789", workPhone: "321654987"};

(And the next time that object goes across the network, my libraries will be nice enough to put the quotes around the keys for me.)

That’s not at all strange – the type is variable and can be specified by user. Not to mention you can have more than one home or work number. You sure can serialize your data to your keys, but then that would be rather strange.

Anyway just imagine there’s an extra field, problem solved:

    <phoneNo type="home" priority="1">…</phoneNo>

Not sure why you'd edit JSON by hand in general. It doesn't tend to be used for configuration files, AIUI this move is following the trend of sending JSON over the web. So surely looking at parsing JSON vs. XML in a browser is where the real comparison is?

This is the most peculiar thing about the JSON mania of all. I mean your web browser programming consists of 99% DOM manipulation and suddenly everyone forgets how to do that when it comes to making Web API calls. Maybe we need jQuery for XML?

My programming consists of 99% Javascript, ie, manipulating Javascript objects. The data structures (ie objects) translate flawlessly into JSON (I just gotta remember the quotes around the keys, is all...). Switching my brain from Javascript to the-data-across-the-network (and back) requires practically no effort at all, because it's the same thing.

EDIT:

> Maybe we need jQuery for XML?

Adding more layers of complexity to deal with complexity? I am further convinced that XML is bloated and bureaucratic. I am so happy that modern browsers let met get away from jQuery. Plain, vanilla JS is better for the soul.

I don't have an answer but I do have a follow-up question.

There are no pre-built GChat clients and I have been planning to create one using Phono, which is built on jQuery. Are there better solutions other than this, or the linked example Superfeedr?

Google Chat is my main IM client and I know it's built on XMPP. I just want to pick a solid JS library to build with and make a web interface that can work for people.

You can also use something like Strophe.js to build your own client. Or you want to add an XMPP/Jingle support as well? How exactly would it work, through routing Jingle over WebRTC?

Update: I just looked at Phono - it doesn't seem like it supprots XMPP/Jingle, even though it uses WebRTC. So this kind of defeats the purpose, since you can't initiate audio/video calls to other XMPP clients.

Can you advise any combinations of JavaScript client libraries and XMPP servers (i.e. without a need to use third party gateways) which can already normally support XMPP over WebSockets and Jingle over WebRTC?

XMPP over Websockets is implemented in various servers:

  - Prosody (http://code.google.com/p/prosody-modules/wiki/mod_websocket)
  - openfire (http://code.google.com/p/openfire-websockets/)
  - Tigase (built-in as far as I could see)
  - Ejabberd in the Commercial version, the opensource version doesn't really have one yet.

Regarding Jingle over WebRTC: Various people are working on it (or have been, in my case) and it's getting closer to workable. What I encountered, when I tried ~a year ago to do it, was that not all SDP messages could be easily transformed into the Jingle format. Might have been changed now though as the WebRTC Specs have changed quite a bit.

Thanks for the pointers. I hope open source ejabberd will catch up, since I prefer it in general. Are any of the Jingle through WebRTC projects public to track their progress?

I agree. It's better to help improving current XMPP libraries for good support of WebSockets and etc. instead of bringing JSON into the picture.

There are a couple of key points here which a lot of you are missing which are:

* The JSON -> XML conversion is done on the server rather than in the client. Browsers are surprisingly bad at xml (one of the reasons buddycloud chose to move the xmpp layer out of the browser), and xml dom manipulation can make for overly complex code (we should also bear in mind one of the philosophies of xmpp which was simple clients, complex servers), and this leads me to my second point:

* It's really simple. This opens it up as a no-brainer solution for web devs who would otherwise just throw up a proprietary silo system based purely on socket.io or similar. Sure, if you really want all the complexities and nuances of xmpp then use an xml based library, but if you just want to send simple messages between users then this is really simple, and hey - what would have been a silo system is suddenly federatable at zero cost.

The point here is that this library is not aimed at hardcore xmpp devs, but devs who would be put off by anything other than json. Make this library as easy to use as socket.io on its own and suddenly you've got more systems running on xmpp which wouldn't otherwise have been (which is great for everyone)!

Is it XEP-0295 compliant?

http://xmpp.org/extensions/xep-0295.html

All this does is wrap all of the XML in a {"s": "<stuff here>"}. That is useless.

XEP-0295 is humorous (1st april joke).

Really? Is that what the green text up top meant?

What is so scary in the XML?? I didn't get the point of this.

Personally, it's just another ML I need to think about.

If it's JSON, I can at least pretend that my data structure stays coherent from A to B (ex: node.js webapp to js frontend).

It's also easy to look at, appearing much like a dictionary or hash + array set in many programming languages.

XML is a "must know" kind of thing for any serious developer anyway, so I don't really see your point. XML and JSON have enough different use cases, even though they have common ones as well. For starters, XML can be automatically validated when needed using schemas, while JSON can't (even though there is a draft for JSON schema in the works). So XML has advantages over JSON is various scenarios. The price is more complexity naturally, but one should expect that.

But besides the point, XMPP already uses XML. So there is no need to reinvent the wheel here.

While, mostly working in client development, I'm a professional developer of 12 years, and while I've had my random encounter with XML, I've managed to avoid it, and do not "know" XML in any significant capacity.

It's easy to forget that there is a large set of different developers. And some don't know what you may consider common knowledge.

IMO, something being JSON over XML would be a significant plus in my mind.

Working for a company that has started adopting JSON Schema as our primary validation tool, I can definitely agree that XML still has the upper hand when it comes to a seamless experience in data validation. Schema validation is still really primitive (though with the love JSON has been getting in the past couple years, it shouldn't be too much longer until it's in fighting shape).

Something about XML just has never really sat well with me - whereas JSON just works when it comes to actually modelling objects in a way that makes sense and can be picked up quickly. I can't say I've yet to find a place where I absolutely needed a feature of XML that I couldn't find in JSON, but would love to know more situations where I can be proven wrong.

I think ultimately if the goal is just to JSONify the world, then that's no good - but if we're looking to make protocols more accessible and hackable, JSON may be the way to go.

I wasn't referring to knowing it, I was referring to having to _consciously_ wrestling with it.

With regards to schema/ validation, wouldn't it be a poor choice to put this logic in the transmission layer?

When mentioning validation I compared them in general, not in the context of XMPP specifically. Validation is always expensive, and is used only when really necessary.

Reply on news.ycombinator.com