Hacker News Reader

Comment by tannhaeuser | Hacker News Reader

I'm not very much into chat apps and protocols, but isn't it possible to just revive XMPP? Ten years ago we used Adium and Pidgin, and federated chat seemed like a solved problem. I understand XMPP with its extensions and use of XML fragments might be complex, but it sounds like matrix or another new protocol isn't easier. In a way, those willing to invest time into F/OSS chat apps have done the best they could with XMPP, and creating a new community around a protocol for no good reasons seems like a wasteful thing to do. XMPP folks shouldn't stop using it just because of Google's kiss of death.

View on HN

Why are people so attached to XMPP?

There was no entity preventing XMPP from thriving, if the community really wanted XMPP, we would have had XMPP everywhere.

But honestly, as someone who ran and tried to use XMPP for many years, the entire protocol was an unmaintained divided mess. And the XEPs didn't help, they felt like hacky workarounds to keep an old protocol up-to-date with the modern needs.

Matrix feels very consistent, specially in the client side. I have all the modern features one could have wanted (E2E encryption, video and audio calls, file sharing, proper synchronization across multiple devices, etc) working out-of-the-box across all my devices. This was almost impossible to achieve with the XMPP clients existent nowadays.

I have very same impression. XMPP is basing on XML where parsing is order of magniture more complex than JSON. In lot of ways very similar to SOAP mess where you could not comunicate across different software because each of them supportet different set of SOAP extensions and good luck with communicating with web client.

> XML where parsing is order of magniture more complex than JSON

You're supposed to use an off-the-shelf XML/JSON parser, not write it yourself.

Even if you don't parse it yourself, XMLs data model is usually more of a chore to consume.

XML/SGML is highly appropriate for representing rich text, though, which is the purpose of chat apps/logs. Using JSON you'd have to invent ad-hoc markup on top of JSON, loosing any advantage it may have.

> Using JSON you'd have to invent ad-hoc markup on top of JSON

That's basically a description of ActivityPub that wraps HTML in JSON!

Will activitypub let me do:

  <body> markdown text </body>

And ship some js to convert to html?

No, each server has their own rules what and how will they present content. Usually they accept limited subset of HTML and filter out everything else.

Nobody uses rich text formatting, and even if they did, the message body is only a small fraction of the total protocol content.

Where does "order of magnitude" come from? In my experience, XML parsing is about three times slower than JSON and around two or three times the size. For most complex applications, this is not a show-stopper, because the bottleneck is elsewhere.

XML was a hype in 1999, when XMPP has been invented and JSON was a hype fifteen years later. When people invent protocols they just use what is trendy in the moment.

XML is not popular because of JavaScript developers.

Coming from .NET, I do not care using JSON or XML, because the APIs System.Xml.Linq, Newtonsoft.Json and the upcoming System.Text.Json are a joy to use. I think the lack of good native representation of XML in JavaScript made it a unwanted choice.

Regards SOAP I tend to agree to you. Beyond the Basic profile it was a hassle to interop.

>XML is not popular because of JavaScript developers.

This could not be more false. JavaScript is built around the DOM. You can parse XML with DOMParser and query it with the same interfaces that you build your UI with. The X in AJAX is XML. The "good native representation" of XML in JavaScript is the very core data structure that JavaScript was designed to interact with.

XML is not popular because it's complex. Escaping is hard. XML bombs make it unusable in its raw form.

Fundamentally, XML doesn't give you data structures that look like the ones you use in your application logic. You have to transform your data into something that makes sense in XML, then transform it back. With JSON (and other formats) the data structures that you're given are present in almost every programming language. You don't need to model your data differently for transit or storage than you do in your application logic.

Good arguments but using DOM Api in JavaScript needs many more lines than JSON parsing and access. jQuery made HTML access bearable for a simple reason: DOM sucks :)

Average Joe Dev does not reach the impedance mismatch you mention correctly.

The DOM interface is roughly the same in almost every language. The JSON interface is almost always simpler. It's no mystery why JSON is more popular. XML never died because of JavaScript developers.

XML is extremely popular with the JS crowd, they just don't know they're using it ;) JSX (React's template language extension to JavaScript) stands for "JavaScript XML" [1], and there was E4X as an official ECMA spec for XML literals before that (though admittedly not as cool as React is, doesn't have vdom and update events, components for MVC, etc). And JavaScript was indeed invented for DOM manipulation, it's just that the DOM API simply sucks.

The whole XML hate thing is symptomatical for the staged this-vs-that discussion we endure in today's forums.

I guess there are those document representation schemes that everybody hates on, and those that nobody uses ;)

[1]: https://en.wikipedia.org/wiki/React_(JavaScript_library)#JSX

I agree.

E4X was far more powerful than JSX. JSX is just a nice syntax for object construction. E4X could additionally query XML (think XPath but different) not to mention support for XML namespaces, etc.

I agree with the rest though.

> XMPP is basing on XML where parsing is order of magniture more complex than JSON

The last time I heard, there weren’t two JSON parsers that actually accepted the same language, so I’m not sure there’s an existence proof to back this statement up. :-)

XML is more complicated, but if we want to compare apples to apples, it’s only due to the white space sensitivity. (dtd, schema, etc don’t have analogous technologies in the json world, and at least xpath is a standard, unlike jq).

Sure, XMPP has problems, but ZERO of them comes from using XML. Thinking that using JSON will help solve the real problems any federated protocol would face is very naive and will lead to more than a few unpleasant discoveries down the road.

fwiw, nobody on the Matrix team (or that I know in the community) thinks that JSON is somehow fundamentally superior to XML - it’s just a data encoding. More recently we’ve been using CBOR over JSON for instance. Anyone who claims Matrix is about “using JSON rather than XML” to do the same thing is catastrophically missing the point.

https://twitter.com/joepie91/status/1102888957510107136?s=19

Check this active member of the community. Also folks on this very page claim how xml consumes much more resources

Weren't facebook messenger and google talk based on xmpp? I know that at least they were compatible, but in the end they removed that. Trying to force people on their own client

This is the problem - if you have a large user base there is simply no business incentive for federation. It is so much easier (and cheaper) to just build your own silo without worrying about interoperability. Plus you have the ability to monetize a closed system through ads, data mining, etc. There are so many advantages.

And people don't complain (much) because, for chat especially, they have had years of conditioning that leads them to accept (and expect) that communication with their online contacts depends on installing multiple apps from multiple vendors and sharing their data with those vendors on whatever terms the vendors decide.

And don't forget about spam. Spam becomes a problem when your system gets bigger.

Google Talk was, Facebook just had an xmpp gateway.

Google killed the xmpp momentum by gettimg scared of Facebook around 2010. and adopting the 'walled garden' approach.

Of course, app that implements it's only server API would be consistent with itself. You see, Matrix isn't mature enough to have XMPP's problems, such problems must be deserved first.

And seeing how fixated matrix zealots on JSON superiority over XML as a format, I don't think Matrix will ever mature to a point to have problems faced by real federated protocol with multiple implementations of both clients and servers.

XMPP isn't XML. https://about.psyc.eu/XMPP#XMPP_isn.27t_even_proper_XML

What does JSON vs XML have with facing problems related to maturity?

Plenty of us are still working on XMPP, and there are thousands of XMPP servers and many more users on the network.

It's true that since the original Jabber project began in 1999 there was a vision of giving users freedom of software and service, yet that vision is still nowhere near reality in 2019. But we are still an active community and still work towards that vision. XMPP is not dead.

Unfortunately Pidgin and Adium were two of the most popular clients for XMPP, and their development (in Adium's case) and XMPP development (in Pidgin's case) has almost completely stalled, which is unfortunate. Alternative XMPP projects are still very actively developed.

The XMPP community recently began holding sprints, where a bunch of us get together in real life to focus on specific topics, such as UX and interop. It's been great!

For those searching a very simple to use XMPP client:

Quicksy.im: https://play.google.com/store/apps/details?id=im.quicksy.cli...

It is a fork (same developers) of the popular Conversations client, which does not bother you with provider selection and such kind of technical decisions. If you just want to try XMPP or invite someone, it is probably the easiest way to start with.

> and there are thousands of XMPP servers

proof please. Checked the top 3-4 xmpp servers last summer and they almost died.

IRC looks 1000000x more active.

I'm not sure how you are judging "activity". I suspect you are talking about chatrooms. In this sense, yes, IRC probably would look pretty active - it is built around the concept of channels, the majority of which tend to be public. XMPP is far more diverse, rooms are spread across multiple servers with no central way to list them. A project that aims to spider the network and help find rooms is at: https://search.jabber.network/ (note that this doesn't currently index jabber.org, traditionally one of the larger hosts, due to a technical glitch I believe).

My mention of thousands of servers is not related to public chatrooms however. There are a bunch of public servers which you can register accounts on, but many more private ones (that federate) run by businesses and individuals and other organisations. A quick shodan search reports 55,760 unique IP addresses speaking XMPP publicly on port 5269 (the default federation port): https://www.shodan.io/search?query=port%3A5269+http%3A%2F%2F...

> I'm not sure how you are judging "activity"

OK, "324 Kuketz-Blog (kuketzblog@rooms.dismail.de)" seems like most popular jabber channel, right?

Here's the #freenode top in this moment:

#linux 2000

#python 1950

#archlinux 1860

And it's ONLY first 3 top channels.

So, the facts matters.

> the facts matters.

Indeed. Could you also provide the number of 1:1 chats and private groupchats running on freenode, then? Both of these are way more popular use cases for XMPP than public rooms.

XMPP is different from Matrix.

On Matrix practically everyone is matrix.org. The other public servers I tried are relatively slow and suffer hickups when joining rooms.

On XMPP, many people run servers in their companies, projects, or homes. Which are the "top 3-4 XMPP servers"? I probably do not even know them.

But you are right: XMPP is used mainly for IM, i.e. one-to-one dialogues or smaller, non-public chatrooms. I'm not aware of any huge, public chatrooms in XMPP, and for that I'm on IRC — via a gateway from XMPP called biboumi.

We have almost finished a pretty decent XMPP to Matrix bridge, if you are attached to XMPP (https://github.com/matrix-org/matrix-bifrost with a bridge running at bridge.xmpp.matrix.org). However, Matrix has a fundamentally different architecture: replicating conversation history over a full mesh of participating nodes, rather than passing messages via a star topology, and we opted to build that as a fresh stack rather than creating a mutant hybrid of XMPP.

I'm so happy this bridge exists. I'm using it to join rooms of some open-source projects that opted to use Matrix, and I know of some people on Matrix using it to join similar rooms on XMPP.

Meanwhile both sides get to use free and open communication protocols :)

> isn't it possible to just revive XMPP?

Please no. XMPP has adapted over time to support various chat and instant messaging features that we today take for granted. These adaptions come in the form of Extensions[1]. A year ago I spent about two hours with a geek friend to configure something that resembles what we get from any other instant messenger today. That's not acceptable to the outside world.

Meanwhile, Matrix just worked. Granted, UX involved with the key management associated with E2EE was not what it should have been. But, that has largely been fixed in v1.0.

Funny thing, Matrix is still struggling (like every other federated network) to explain the concept of federation. People may have grasped email, but for some reason it seems hard to translate that knowledge to other federated technologies.

To suggest that we should just go back to XMPP seems like insanity to me. Matrix stands a chance. It's freaking wonderful!

[1] https://xmpp.org/extensions/

> Funny thing, Matrix is still struggling (like every other federated network) to explain the concept of federation.

Are there any data on how federated is Matrix? I mean number of users on different servers.

From my experience most of people are on matrix.org server and even if they run their own homeservers the identity server is still vector.im.

From memory, about 50% of the users visible in Matrix are on the matrix.org server, which is one of about 30K deployments total that we're aware of, directly or indirectly. This will likely change in the near future as the French deployment expands, which should dwarf the matrix.org server. In the medium/long term, we'd like to at least disable registration on the matrix.org server if not turn it off entirely (once decentralised accounts land).

The identity server is used ONLY for mapping email addresses and phone numbers to matrix IDs, and is strictly optional, so I'm not sure it counts as centralisation. We are also hoping to remove it entirely from the mix in future (switching to storing 3pid->mxid mappings per room instead).

Thanks for your post. I've been wondering about federation distribution in various protocols since seeing similar stats for Mastodon [0]. Having federated protocol where vast majority of users still use one server is sub-optimal IMHO (this is not only for Matrix).

[0]: https://rosenzweig.io/blog/the-federation-fallacy.html

As for identity server, it was just surprising for me (from user's perspective) that when my friends ran a homeserver I still needed a lot of config on Riot for Android to connect to (homeserver URL, identity server URL). I've seen some well-known URIs [1] but sadly Riot for Android did not query them for my e-mail.

[1]: https://f-droid.org/.well-known/matrix/client

Thanks for your work on Matrix, glad that you guys are pushing the boundaries of UX of open-source software!

the mobile clients lagged a bit on .well-known support, but it landed over the last few weeks: https://github.com/matrix-org/matrix-ios-sdk/pull/644 and https://github.com/matrix-org/matrix-android-sdk/pull/436. Hopefully this should improve things a bunch.

For 'the federation fallacy' thing: our approach is to first decentralise accounts (so you can transparently move them between servers), at which we can turn off matrix.org (or at least disable signup) assuming there are enough other quality public servers available to incorporate the resulting diaspora. In the longer term, we want to enable P2P in Matrix so you don't even need a server at all if you don't want it, at which point the federation fallacy argument entirely falls apart :)

and thanks for the thanks! :)

Doing the same for XMPP: http://upload.zinid.ru/xep-eax-cir.html

Let's compete.

The problem of XMPP is just lack of good clients on some platforms, most notably for iOS. For Android there is an excellent Conversations.im client [0] that I've successfully use with my non-technical family and friends. (There is also Quicksy [1] that uses phone numbers for onboarding but it's XMPP all the way down). Conversations doesn't use a lot of battery (anecdata but it's 1% on my phone) and just works.

[0]: https://conversations.im/

[1]: https://quicksy.im/

As for servers ejabberd is successfully used from small deployments to big MMORPGs. The team is extremely focused on performance and from what they say XML isn't actually a big performance problem. Prosody, another server, is in my opinion also high quality, even though it's developed in free time (AFAIK).

The "myriad of extensions" is mostly solved problem due to Compliance Suites and automated testers [2] (think SSLLabs) that people use to get their servers in shape.

[2]: https://compliance.conversations.im/

For the record I've used Matrix for work too and it's not bad, the clients are decent (I've used web client), but it's not a revolution especially if one's contacts can use Conversations, that, in my opinion, is just very good.

I have to agree and disagree. I agree, that XMPP clients are not very good. And horrible on iOS (and MacOSX). I disagree, that Matrix has good clients. Riot is an OK web client, but too slow and too confusing for my taste. And native desktop clients, like Nheko or Quaternion, are not yet very useful. XMPP has many more mature clients at the moment.

In case of iOS/macOS - there are: * BeagleIM by Tigase (https://beagle.im/) for macOS * Tigase Messenger for iOS (https://itunes.apple.com/us/app/tigase-messenger/id115351683...)

I am completely on your side. Matrix did improve upon the state of XMPP in 2012. But XMPP improved too and therefore the problems we had 2012 are not relevant anymore today.

There is just one big problem with XMPP nowadays: Very few people are interested in XMPP as everybody wants to learn the new shit aka Matrix. And actually, that is the one thing I blame the Matrix devs for. Instead of improving an existing open IETF standard they decided to invent something new and split the community.

Technologically, Matrix is neither better nor worse than XMPP, as far as I can tell, but splitting the community wasn't such a brilliant idea.

Well it is better in some regards, such as each server being an independent store of events of all rooms in which its users are participating.

Nobody buried XMPP. Write good server and client, comparable with proprietary protocols by features and it might work.

Nice, if XMPP works for you, but I hope do not use Pidgin or Adium nowadays. They still work, but AFAIK, they do not support newer XMPP features, such as encryption using OMEMO, HTTP upload, message archive management, etc.

There are nice, modern XMPP clients like Conversations on Android, Gajim on Linux and Windows, etc. that do support the latest XMPP features and are much more pleasant to use today.

Matrix is the perfect example of xkcd 927 - https://xkcd.com/927/

They built something like XMPP, just a bit different. What they got is more fragmentation in the space of free messaging. None of the justifications they have how their approach is different justifies the costs of additional fragmentation they create.

This is super frustrating, people don't seem to have an understanding what the costs of incompatibilities are. You can also observe pretty much the same at the micro level of within XMPP and encryption standards, where everyone seems to think the solution is yet another encryption standard instead of having just one.

I'd counter and suggest that the super frustrating thing is XMPP zealots saying "oh my god how dare you try to create a different protocol". With this mentality, nothing would evolve and we'd be stuck on svn rather than git...

To reiterate once again, the things which Matrix tries to do differently to XMPP include:

* Be a replicated conversation database for realtime conversations, more like usenet (NNTP) if anything than IRC or XMPP or SMTP (or, alternatively, an open standard alternative to Slack, which is also focused all around syncing conversation rather than passing messages).

* All conversations are group conversations. There are no such thing as 1:1s; just a room with 2 people in it.

* To be clear: there are no APIs to send a message from one user to another in Matrix. Instead, you sync a message to a room, which updates its conversation history, and the history update is then synced to all the other servers (and users) in the room.

* Have a single monolithic versioned protocol (https://matrix.org/docs/spec) rather than a finely granular cloud of XEPs which may or may not be compatible or best practices or implemented at any given point.

* Build in E2E encryption from the outset.

* Implementation before formalising spec

* A "there is not more than one way to do it" mentality on developing the spec (i.e. a python rather than a perl approach)

Meanwhile, we have spent a lot of time writing decent bridges to ensure that we do not fragment communication with chat systems - be they open or closed. For instance, right now, I see 41 people participating in Matrix HQ from XMPP, ~200 from IRC, ~40 from discord, ~20 from telegram, 5 from slack, and the rest (~3000, including idlers) from elsewhere in Matrix.

> we'd be stuck on svn rather than git...

Don’t you mean CVS?

> I'd counter and suggest that the super frustrating thing is XMPP zealots saying "oh my god how dare you try to create a different protocol".

I see no such consensus within the XMPP community, but yes personally I agree with those zealots. My interest in federation is making it possible to communicate with anyone just like I can call anyone by phone. Inventing an incompatible protocol obviously won't help with this goal, even if you ignore the issue of splitting up scarce manpower and assume bridges can act as a partial workaround (at least for geeks who can cope with their limitations).

> With this mentality, nothing would evolve and we'd be stuck on svn rather than git...

Comparing XMPP to SVN and Matrix to Git just reads like trolling to me. The obvious difference is that XMPP is highly extensible while SVN is not at all. So the discussion is not about whether or not to evolve, the question is whether evolving federated IM required starting from scratch, i.e. whether the advantages are worth the resulting fragmentation. And I just fail to see the problem in using/extending the XMPP spec to replicate your DAG.

> Have a single monolithic versioned protocol rather than a finely granular cloud of XEPs which may or may not be compatible or best practices or implemented at any given point.

I totally get where you came from, the fragmentation within the XMPP universe is annoying. The problem is that a monolithic spec doesn't solve this issue at all. It's not like code magically updates itself when you update a monolithic spec. The problem is missing manpower on the implementation side, and ditching modularity from the spec just makes it harder to cope with this in graceful ways.

I'm not sure that the problem is missing manpower on the implementation side - it's more that it's hard to know as a developer which current blend of XEPs is the recommended combination and which might have the most chance of working between a given client & server (and server & client) combo.

Things like the XMPP compliance suite XEPs have helped a bit with this, but looking at the XEP list and trying to work out which XEPs you should be using on a given day is still daunting - as well as trying to track which clients are most likely to be supporting them.

The idea on Matrix is that you say "Hi, I talk Matrix CS API 0.4" and be done with it - and you end up with much more social pressure to keep up to date with the current latest spec, because otherwise you are simply falling behind (rather than happening to chose not to implement some XEPs).

It boils down to a question of governance & social dynamics rather than anything related to writing code (magically or otherwise).

And admins will update their deployments instantly? Because of monolithic core? Interesting...

Yes, instantly. Or else they'll lose connectivity with central matrix server and 80% of users.

> I'm not sure that the problem is missing manpower on the implementation side

If it isn't, why are there so many red crosses on https://matrix.org/docs/projects/clients-matrix and so many servers that aren't even capable of talking to matrix.org?

> why are there so many red crosses on https://matrix.org/docs/projects/clients-matrix

Because the world is littered with unfinished projects? The key thing being that the authors KNOW very concretely that the project is unfinished, and there is a big red X next to the feature in question.

> so many servers that aren't even capable of talking to matrix.org?

Because we only released the first stable release of our federation API a month ago?

I used to be a big fan of XMPP, but I've also come to the conclusion that it failed.

Technically, too long the clients just weren't good enough. First the desktop ones, then the mobile ones. Not losing messages if you were logged in with multiple clients was hard.

Matrix (after waiting a little) now just works. Yes, maybe I just don't have any energy left to fight on this front, but for a huge chunk of people I communicate with it has finally replaced IRC. Finally as in reuniting the people who had stopped and also didn't want Slack or Discord. And never got on the XMPP train.

Now we'd just need more native desktop clients, but so far Riot is good enough for me.

I still use pidgin for my "eve online chats" and it continues to do all the things I need pretty effectively. I've had a couple of slacks and a couple of discords over the years and they really haven't improved on many axes at all.

Reply on news.ycombinator.com