Is XMPP any good? Also, let's write a client in Tcl, maybe
wusspuss.neocities.org
wusspuss.neocities.org
It would be just as easy to write a post like this about JSON, Matrix and so on. Both have many warts.
But I really don't much care. No technology is perfect, and there will always be people who dislike choices that other people have made. What matters is what we do with the technology. XMPP has proven itself able to evolve over more than two decades, during which the internet and the way we use it has shifted dramatically, and yet it continues to be relevant as new people discover and adopt it.
It's unfair to describe things like multi-device sync as "afterthoughts" when XMPP was one of the first real-time messaging protocols to have support for multiple concurrent connections of the same account built in, at a time before the iPhone was released, and most people had a single device with wired access to the internet only.
Support for multi-device sync is part of the suite of protocol features required by modern XMPP implementations, these requirements are updated every year to account for the changing technology landscape: https://xmpp.org/about/compliance-suites/
I personally like the simple delivery model of XMPP compared to the "distributed database" model that Matrix pursues. I think there are use-cases for both approaches. Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort. What's important is that both provide openly-documented protocols with open-source reusable implementations, and (in my opinion) most important of all: decentralized open communication networks.
The problems facing the broader adoption of open networks for modern messaging are not protocol problems.
About the XML bloat you've mentioned: The problem isn't just the wire size, it's working with it as a developer. XML is needlessly complicated to deal with in code and super painful to read when debugging. Tooling for it seems lacking too; I even found that iOS's built in XML parser was very slow at parsing large elements, with time/mem used seemingly growing O(N^2). Found that out when the XMPP-using startup I was working for decided to send all their XMPP IQ messages as just wrapped JSON payloads.
About extensions: As cool as those XEPs are, it makes for a fragmented federated ecosystem when so many basic features are extensions. It's not awful when maybe you can't send large files on one server, but when something like auth is broken...
For messaging purposes, it is absolutely identical to JSON in this regard. Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.
Understand this: xml is meant to be read by computers, not humans.
> The criticisms linked are all valid, and there are good reasons it's being abandoned
It's 2022 and there is still nothing better for federated messaging.
There's no "JSON mindset" to produce anything comparable to this crime against bandwidth and code brevity:
<stream:stream from="draugr.de" id="{ID}" version="1.0" xmlns:stream="http://etherx.jabber.org/streams" xmlns="jabber:client">
>Once you have many existing implementations out in the wild that need to interact with each other, you'll need namespaces or their NIH analogue. And JSONs with namespaces will be just as bloated as XML.
Even XMPP, with XML, could do just fine without namespaces. The only place where namespaces are the differentiating factor in practice is <query/>'s and <x/>'s. Couldn't they just be be called <mam_query/> or whatnot? Oh, and namespaces specify the version. Like for ack's they specify the version 3 every time. But it's something that has to be (and is) clarified when enabling said acks anyway.
Online sources say it's meant to be both human and machine readable. It would be less bloated if it were just for machines. Seems like a generalization of HTML, which is meant to be hand-crafted in some cases.
Even if it were for machines only, it'd still need a human-friendly way to manipulate and debug with, and there isn't one. Protocol buffers have the text version for humans and the serialized binary for machines, plus the generated getters/setters for code. XML is a pain to deal with no matter how you abstract it.
JSON with namespaces would just mean another key-value pair in the dict or something. But I'm not sure why that's needed anyway.
> It's 2022 and there is still nothing better for federated messaging.
True, but federated messaging isn't in style anymore, and honestly I hardly even used XMPP. Was working on my own protocol 2 years ago that I truly believed was the best, and had a working reference impl, but had to abandon it. Got 2 jobs already.
I work with XMPP for many years and approximately zero problems we face would be solved by switching to JSON or whatever.
This "JSON is better than XML" is a modern version of old religious debates like Gnome vs KDE, Pascal vs C, Christianity vs Islam.
I think most people agree that XMPP messages are hard to read during debugging, and this is somehow not a problem in any of the JSON-based protocols.
XML is not a problem. Parsing it is not a problem. Reading it is not a problem, too. None of the problems ever faced by us in the development of XMPP software would have been solved if it was switched from XML to JSON, nor made even a slightly easier to solve.
That is untrue. Matrix is undoubtedly much better.
Tying identity to a server is insane in a decentralized environment. Decentralization is all about keeping Google sized companies from controlling things. But small companies come and go all the time, and nobody trusts them to be around in 10 years.
They aren't solving the SSL problem though. Mumble has a good solution their with their trust on first use connections. Self hosted really should ideally not need a domain name if you want individuals to host it. It should be something you can literally buy a $20 preconfigured hosting node, plug it in, and go.
Tox seems to be most promising of current ones, it just needs to fix it's mobile data use issues.
This issue you are talking about is reliably solved ny owning your own domain. You can self-host it or use the commercial provider for it, either way you can change provider and retain your address.
And regarding these matrix guys, they are so overtaken by a NIH syndrome [1] that they couldn't even follow a common URI syntax. Are you really sure they are best fit to develop standard protocols of any kind?
There's very few projects that look at all interesting since the BitTorrent days, most everything is blockchain focused, and aiming to completely replace centralized at all costs with no ability to use central servers, even if it performs badly.
Owning your own domain has some disadvantages though. Especially with traditional federation. For one thing, your RasPi at home is probably not as reliable as Google cloud, and neither is a $5 month VPS.
Now you've got a single point of failure, that's on consumer hardware, with no professional IT staff managing it. I don't exactly want to pay for something that's less reliable than Gmail and WhatsApp, since an outage at the wrong time could hurt me way more than Google spying is likely to.
The only way I think self hosted makes sense for a typical consumer is if it can seamlessly transition to fallback servers, and you can talk offline on a LAN, and it doesn't rely on domains.
Like, I should be able to put a DHT record that says "I have linked accounts at these servers, try mine if you can, but if that fails here's a backup that we can meet at, that forwards to my main server".
That's what NIH syndrome does to people.
Maybe judge not by reading protocol description but by trying and using said protocol, like, chatting with people a lot? With the latter approach you'll notice a very handy feature of Matrix - not losing messages.
Not losing messages is also a feature of XMPP. In the distant past, especially before stream management, various kinds of network issue could certainly cause lost messages. These days, almost all implementations support the necessary features to make lost messages virtually impossible (of course, being federated, it's always possible that e.g. the remote server is permanently shut down just after successfully accepting your message, but that risk exists in all protocols including Matrix.)
Regarding the standard extensions, the most common way to prevent message loss is by using stream management, however there are edge cases in SM when the messages can still be lost in milti-device scenario. It is not surprising, SM was not meant to prevent message loss per se, just to resume a stream.
The main problem of XMPP on modern mobile devices is this: XMPP is designed to be always connected, while modern OS shut down apps when they are in background. So to make XMPP fit in the modern world we needed to make a transition from a resumption of stream to a synchronisation of state.
You're right, Matrix immediately stands out as what XMPP should have been from the beginning.
Hi, I've worked on client and server implementations of both XMPP and Matrix. This is a wildly inaccurate - but unfortunately common - view of things.
It's correct that Matrix is complex to implement server-side, more complex than XMPP, and it's also correct that it does a lot of things very differently from older protocols like XMPP. But here's the thing - that's a deliberate choice and it is for good reasons, not just a case of ignorance or "bloat" like you seem to be implying.
(Sidenote: if there's one thing I've learned over the years, it's that somebody talking about "bloat" is a huge red flag that they don't have a very deep understanding of something. If you have a genuine complaint about unnecessary complexity, you should be able to point it out specifically, instead of a handwavy term like that.)
Anyway, to get back to the point... much of the complexity in Matrix comes from a single design objective that differs from XMPP: it implements decentralized rooms instead of centralized ones. That is, rooms are not dependent on a single server for their continued existence, they will remain available regardless of server availability.
This might seem irrelevant, but it solves a huge problem that XMPP has been struggling with for years, and that IMO is a major reason why its MUCs never became widespread: the problem of putting trust in a server operator for the continued existence of your community.
Every single time you create a MUC in XMPP, you are essentially betting on the continued existence of whatever server you are creating that MUC on. If it ever goes away, so does your community - and unlike with a list of private contacts, it's hard to rebuild a community. That's a tremendous cognitive overhead for starting a community! And one that most non-techies - understandably - do not want to have to deal with. Trust is hard.
Matrix doesn't have this problem because of its decentralized rooms; it doesn't matter how many servers go away, as long as the homeserver of at least one room participant still exists, the room continues functioning as it always has. That means that you create a room "on Matrix", not "on a specific homeserver". That's much easier for users to deal with.
The tradeoff is that unlike with XMPP, where you have a single authorative server for handling things like access control and moderation and membership, this needs to function entirely decentrally in Matrix - so now, the network is suddenly a distributed system that needs to handle state synchronization and everything that that entails.
And that is where the server-side complexity in Matrix comes from. It is necessary, unavoidable complexity if you want to solve the problem of "where do I put my community" in a way that actually works for regular users. It sucks, but there simply is no simpler way to solve this problem.
The reason XMPP doesn't have this complexity is because it doesn't even try to solve this problem. If the rumoured decentralized MUCs ever become a thing, they will involve similar complexity.
Edit: Also, it would be nice if y'all recognized that the priority should be to get the general public over to open comms systems, instead of constantly trying to start wars against Matrix folks. The constant loud misinformation and attacks are getting very exhausting.
I agree with everything you said about decentralised MUCs, and for the "communities" [1] use-case it will be great when XMPP eventually gains this capability (sadly, a lot of effort in IM protocol development has been split to a shinier new thing these days, so I guess progress is slower everywhere than it could be).
I also agree that decentralised MUCs are inherently complex. What I don't agree with is all the other reinventing of the wheel in Matrix.
If your claim is that every decision to start from scratch (leaving aside decentralised MUC) in Matrix's protocol design was based on well-founded technical reasoning rather than either deep-seated NIH syndrome or a desire to build from scratch to maintain more control / extract more value (I strongly suspect the latter), then you should be able to point to that reasoning — these decisions were discussed in the open in a standards forum, mailing list or the like, right?
[1] My use case for XMPP has always been more like an open WhatsApp/Signal replacement than an open Discord replacement, where the largest MUC is a few people who all know the server operator personally, so I tend not to notice this missing feature — but I completely agree that it is a missing feature.
Yes, we have always been completely transparent in why we (the Matrix team) built on new foundations rather than extending XMPP. (In practice, laying out our reasoning hasn't been productive however, as XMPP folk just see it as an attack). We started Matrix in 2014 after a few years of shipping XMPP-based messaging apps for telcos (ejabberd + XMPPFramework + Smack + xmpp.js as a stack). Some of the main reasons we chose to start over were:
* We wanted to provide completely different primitives: replicating room history & state rather than passing messages; HTTP RPC as the primary transport rather than XML streams. We wanted to remove the concept of DMs entirely, and instead model everything around replicated chat rooms. We wanted it to be impossible to communicate without sharing ownership of the conversation between the servers. All these are fundamental architectural changes.
* We wanted a single monolithic spec that defined the features available in the protocol for a given version. There should only ever be one official way to do each operation, rather than competing alternatives. (Matrix is still completely extensible though: anyone can propose new changes to the spec as Matrix Spec Changes, and all the APIs support extensible data. However, which MSCs make it into the spec itself is up to the Spec Core Team - a 50/50 mix of original Matrix core team members and folks who joined subsequently from the community).
The reasons were categorically not about "extracting value", but trying to build a successful protocol for everyone who wants to use Matrix - whether that's the core team who nowadays work at Element, or any of the hundreds of other independent companies building on Matrix. It's true to say that we wanted more control, though, given that in order to produce a single monolithic spec document you necessarily need more coordination over "what the protocol is".
Now, just as you could have built DVCS semantics on top of Subversion (as for instance SVK did) - so you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility. Or alternatively you start a new stack (like Git did) with different governance, and built from the ground up to have totally different semantics and behaviour.
So, to be crystal clear: this was not a case of NIH - but just trying to build something with a completely different design to see if it worked out better. And just as the world can support (say) SVG and Canvas as completely different ways of drawing pretty graphics in web browsers, then so can the world support both XMPP & Matrix as completely different ways of doing open real-time communication on the net.
Edit: and in terms of our transparency and where this was all discussed: Matrix evolves transparently via https://matrix.org/docs/spec/proposals - and discussions of the rationale back in the day can be found in the history of the respective public Matrix rooms such as #matrix-dev:matrix.org. (We don't use mailing lists to develop Matrix; we use Matrix.)
> you could have defined Matrix as a weird & wonderful opinionated dialect of XMPP, defining an HTTP transport; expressing the stanzas as JSON; defining a decentralised MUC, and generally twisting it until it basically looked nothing like XMPP and would necessarily lose backwards compatibility.
All of these except decentralised MUC are implementation details, and relatively minor ones at that. Nobody using Matrix needs to know or care what the underlying transport is [1] or whether it uses XML or JSON. So calling them out as design goals that led to the genesis of Matrix strikes me as slightly strange, since they have basically zero effect on the utility of the system.
Wanting a single monolithic spec is a perfectly reasonable desire, but there's no reason that couldn't be an overlay of the XMPP specs — an agreed set of extensions that all compliant clients implement — giving the best of both worlds. (This already exists for XMPP, in fact, in the form of yearly compliance suites, the most recent being [2].)
Are there other strong reasons why Matrix could not have simply been a decentralised MUC extension to XMPP?
[1] Of course, there has been an HTTP transport for XMPP since 2003, so even if we accept that transport does sometimes matter to the user — e.g. in restricted networks — this wouldn't be a good justification for starting from scratch.
So we actually dropped support for MUCs and created our own Group chat protocol, which even provides a fallback compatibility with the very basic XMPP. Extensibility for the win!
It's definitely not identical. There's one major difference, that rarely seems to get mentioned but seems to be responsible for most of the pain:
JSON maps directly to language-native data structures, but XML does not.
The reason for this is that XML distinguishes between child nodes and attributes, but in the basic data structures that most languages have, there's no way to represent this in a conflict-free manner without doing something like `{ attributes: { ... }, children: [ ... ] }`, which is an awkward structure to query.
The result is that when you're working with XML, you almost always end up with a "data API" that's designed specifically for XML (eg. classes representing nodes with getters for attributes and children), whereas with JSON you're generally working with simple nested lists and structs.
This introduces relatively much complexity in what's a "hot path" in terms of development effort - every single time you want to interact with the deserialized data (which is all the time!), you have to deal with an XML-specific data structure instead of treating it as "just another source-agnostic pile of structs/objects".
The result is tightly coupled code that requires the reader to be familiar with the specific XML API/representation being used, instead of it 1:1 mapping from the serialized data to a language-native data structure. That's not good!
While it is true that you are likely to implement a namespace-like mechanism in JSON as well, for open protocols, the difference is that the format itself does not specially define namespaces - which means that the 1:1 mapping always remains true, and it's the protocol developer's choice to eg. reserve a `children` property to contain child nodes, with the understanding that that property then cannot be used for other things anymore.
XML deserializers cannot do this; it's entirely possible for XML to exist that looks like `<tag children="yes"><subtag/></tag>`, and therefore there is no possible 1:1 mapping that a deserializer can use that's guaranteed not to make certain data inaccessible. And since child nodes are not represented as an attribute in XML's data model, you cannot expect protocol designers to avoid a specific attribute either.
Bottom line: I suspect that this seemingly small design difference has far-reaching second-order effects in how deserializers are implemented, and that that is responsible for most of the pain that people experience in working with XML.
So what? It is irrelevant. In all the problems we have ever faced when creating federated chat applications on various platforms, I can't recall a single time when stream format made any difference.
For example, making your messages appear in consistent order on all of your and your chat partner's devices is a real problem. The development of the efficient strategy to achieve this has nothing to do with format, yet, has a much more visible impact on user experience. In fairness, the debate of what is better suited for the protocol development, Message Sequence Charts [1] or Sequence Diagrams [2] has more sense than debating about XML vs JSON.
Plus it's confusing. You have to look through feature matrices instead of just saying "Oh this is version 4, it has everything I need".
You probably have not yet written any JavaScript or CSS if you think that's how things work.
Most of the new stuff is invented by Google and rejected by Mozilla and Apple.
If you are in an intranet environment where you only ever target Chrome and Chrome Mobile and pretend the rest don't exist, it kinda does work like that.
The moment you ask for reliability it stops being simple. Stream Management + Stanza Ids isn't simple, and it shows - XMPP is the only thing I use today where it occasionally happens that a message isn't delivered and I'm not even notified about it.
On a side note I don't believe that this can't be fixed. Maybe if we had end clients checking directly with each other what's been delivered and what hasn't, like actual humans end up doing anyway, I believe it'd be more reliable. Besides instead of relying on the c2s-s2s-s2c cooperation chain to work, it'd rely on just c2c cooperation, giving much fewer variations to test. Just to clarify I'm not talking about a p2p connection but rather e.g. checking the number of stanzas not with the server, but with the interlocutor's client, through the normal XMPP means, like <iq/>s maybe
The last gap I'm aware of was that if you sent a message to someone on a remote server, and the remote server couldn't be connected to, your server would generate a delivery error (i.e. not silent). But* if you went offline before this delivery error reached you, you would never see it. These days such errors are stored in the user's message archive, and available to all your clients for synchronization purposes (so all of them can see that the message failed to deliver).
So with that said, I'm very interested in hearing any details you can provide about how to silently lose messages in modern XMPP. Clients involved, steps to reproduce, etc. Then we can produce actionable bug reports.
[*]: I believe Pidgin is still a notable exception here. Notable mainly because a lot of people still use it, even though its protocol plugin development has stalled somewhat for many years (the project is still being developed, but their focus has been on core/foundation improvements rather than keeping up with changes to modern internet messaging).
Conversations, draugr.de, can't really say much on steps to reproduce - it worked reliably and then it didn't just yesterday, ironically just as I tried to send the article to a friend to check out.
If I understand the two generals' problem correctly, there always is a way to silently lose messages, with any protocol, it's just that some approaches to not doing that are more reliable than others
That's a bit overgeneralized.
A simple protocol for message delivery that will not silently fail: resend it (presumably with a unique identifier of some sort to avoid duplication), pessimisticly and loudly pointing out "failure" (really: ambiguity of delivery), until you get a reciept of delivery.
What the two generals problem points out for this protocol is:
1. the sender of the reciept won't be certain that said reciept was recieved
2. the message may be successfully sent before the sender realizes it
3. if communication is permanently severed, the sender may forever be uncertain if the message ever got through or not (but this can be messaged loudly and pessimistically to the end user)
This creates design problems for coordination of consensus for things like database transactions, but a chat client failing to alert to potentially undelivered messages is just bad, not merely running afoul of some fundamental law of information science.
Of course, it's always possible to successfully send and recieve reciept of a message, only for the recipient's server to suddenly catch on fire and explode before backups are made, thus permanently losing any record of said message, but that's more of a "two generals and an assassin" problem, outside the scope of the original thought experiment. It is also, hopefully, a rarer scenario than packet loss.
So it worked fine for some time, glitched one day, you presumably have done zero debugging as you have no logs or error messages to show us and somehow that means that clients "just lose messages".
I guess you're not using Discord.
JSON has no namespaces, even without taking that into account it's already less verbose, it maps nicely into lists and dicts, I don't know a single JSON parser that can leak the user's IP even as an option, let alone by default (not saying there aren't any at all, just no popular ones), doing a binary version of XMPP based on JSON would take as much as switching over to bencode instead of whatever the unsupported hell EXI is. So no, you can't really write a post like this about JSON. Even if you do, it's not gonna be XML that will be proposed as a better solution.
One thing where XML beats JSON is markup. But with XMPP's unholy approach to markup, it's actually JSON that beats XML.
Causing all extensible JSON-based protocols to eventually reinvent them:
* https://www.w3.org/TR/json-ld11/#compact-iris
* "If new events are added, the event type key SHOULD follow the Java package naming convention, e.g. com.example.myapp.event." https://spec.matrix.org/v1.2/client-server-api/#types-of-roo...
* https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-22... (no aliases, so it's even more verbose)
* https://github.com/OAI/OpenAPI-Specification/issues/1362#iss...
By far the most widely used half-assed solution to namespaces is in the HTML5 parser to special case svgs (and maybe other curiosities? Dunno). And if I understand correctly, that happens to be a full-assed specified algorithm that all current browsers use.
That is all in order to avoid the problems that would have come with requiring the full-assed XML namespaces in that case.
If I were a political operative, I would attempt to bog down the XML apologists on this point whenever they attempt to argue that JSON gets as complicated as XML when you try to include namespaces.
<starttls xmlns="urn:ietf:params:xml:ns:xmpp-tls"><required/></starttls>
There's no reason for acks to specify urn:xmpp:sm:3 each time. Yes, here it at least specifies the version, but that's already negotiated when enabling acks anyway.
There's no reason why <query xmlns='urn:xmpp:mam:2' .../> can't be <mam_query .../>.
Namespaces are a choice, sometimes (e.g. in programming languages) reasonable, sometimes (in a serialization format for passing small texts around) not so much.
It is dictated by a mindset, the XML mindset in case of XMPP, and in the case you bring up apparently by the Java mindset, not to say there's much of a difference between the two
In this example, "mam_" is effectivelly a NIH namespace.
That's funny because it's true. To add to your list: JMAP (https://jmap.io/) has namespaces too.
I'm curious what you base this on. Without a doubt Matrix is gaining traction. It's a relatively newer project, has some new ideas, and there is a lot of buzz around it. But it has far to go before reaching XMPP's levels of achievement - millions of XMPP-connected users via Google Talk, and popular products such as WhatsApp and Zoom built on top of it. That many of these aren't, or are no longer, interested in interoperability or federation is not a protocol problem, and I don't see Matrix solving that.
XMPP continues to be widely used by individuals, organizations and governments who want more control over their communication. Across the XMPP projects and services I'm involved in, I continue to see plenty of new users and developers joining the community.
The reference client/server is definitely a pattern I agree with you on, however. It has served Matrix well, and others too. I've been a proponent of this approach for some time, and detailed my thoughts in the "Products vs Protocols" talk/post at https://snikket.org/blog/products-vs-protocols/
But these aren't the protocol's achievements either. These people have so much money they could make IRC popular.
XMPP has succeeded in its other stated goals of being highly extensible and useful for derivative work, but I think that's in spite of the protocol. There are just a lot of solid implementations out there, like ejabberd (which afaik WhatsApp originally started on, and I worked with it in a startup).
XMPP is a protocol, not a product, so it's not surprising that it doesn't stand up as a product on its own. If someone can come up with a compelling federated messaging product, it's likely that they would base the technical part on XMPP, because virtually all of the protocol work is done, and it's much more elegant and easier to build upon than something like Matrix.
But that's not a protocol problem. The reasons why we still don't have good open federated messaging have absolutely nothing to do with the protocol used for that messaging. They're probably quite similar to the reasons why federation in email has regressed over the last 40 years, from personal/company mail servers back to massive centralisation in the likes of Gmail, despite the protocol remaining exactly the same.
It does have an intended use case, and more opinionation could have improved its chances of achieving that.
> federation in email has regressed over the last 40 years, from personal/company mail servers back to massive centralisation in the likes of Gmail, despite the protocol remaining exactly the same
Protocol not changing is a problem. Email is so outdated nowadays that you're way better off using Gmail for its added security, especially within a company so everything stays within Gmail. IIRC email has changed a bit, but the new security features haven't been uniformly adopted like with HTTPS, so it's too much of a mixed bag.
I've been self-hosting email for ~20 years. The new security features have close to 100% adoption in the real world, especially since most people are using one of the massive centralised email systems, all of which implement all the new security features.
The protocol not changing is not a problem. SMTP is as fit for purpose now as it was 40 years ago, and has been truly decentralised since the very beginning. The protocol is not the cause of the centralisation trend, neither in email nor in IM.
Well that's the thing, if they were self-hosting or using smaller providers, they probably wouldn't all have the new features. And I'm surprised emails from your server aren't being filtered out as spam going into Gmail inboxes.
An open protocol success story is HTTPS. Nearly everyone is forced onto fully-compliant new versions, and self-hosting is more viable than ever.
They are really not difficult to implement. Like, 10% extra effort on top of deploying a mail server gets you all the modern security features. They are really not relevant in the question of whether someone self-hosts or not.
> And I'm surprised emails from your server aren't being filtered out as spam going into Gmail inboxes.
Implementing modern SMTP security features helps with that, and keeping the same server for a long time. In the first year or two my mail used to occasionally get sent to spam (but that was also before several of the modern security features existed), but once your IP has been sending mail for two decades and has never sent a single spam message, you tend not to have a problem with reputation.
> An open protocol success story is HTTPS. Nearly everyone is forced onto fully-compliant new versions, and self-hosting is more viable than ever.
Notably, the decentralisation mechanism for HTTPS and SMTP is identical, and the transport security mechanism is identical. The effort to self-host mail is higher, but that would be completely solved by a modern 'plug & play' mail server implementation. Once again, the protocol is not the issue here at all.
My parents and their friends couldn't figure out Discord after 2 weeks of honestly trying to use it. Biggest sources of confusion seemed to be DMs vs servers, username vs username+number vs nickname, and the sheer amount of stuff onscreen. They reverted to iMessage.
Considering the way in which Matrix spreads unencrypted metadata (even rooms and participants in the rooms, let alone chat events) as far as possible across the network, this traction is quite insidious.
For this reason I'm somewhat uncomfortable joining Matrix rooms over bridges, since it leaks my XMPP address and participation to everyone forever.
A demo is worthless if experience doesn't prove it in the long run. And I have the latter, and it doesn't prove. Yes, I know it's probably me, or more likely my friends doing something wrong. One of them restricted Conversations from running in the background for example, non-techies do all sorts of silly stuff. But then again, it shouldn't break with non-techies either. Nothing breaks with them aside from XMPP. Not even WhatsApp which is based on XMPP afaik
The problems is that many of these modern features were unbelievably late.
I still remember "The State of Mobile XMPP in 2016", https://gultsch.de/xmpp_2016.html
At the time, 9 years after iPhone had come on the scene, most modern features were still experimental draft proposals implemented by no one.
5 more years later there's maybe one client that implements the modern features (Conversations for Android IIRC), and that's about it.
And it's an unknown how many servers implement those features.
...and yet I'm using three different modern clients that work well with each other.
> And it's an unknown how many servers implement those features.
Enough to make me not have to care about it.
It's not: https://compliance.conversations.im/tests/
Obviously a new user is not expected to fathom what all this means before getting started, which is why there are curated lists of public XMPP services, e.g. https://providers.xmpp.net/ and https://joinjabber.org/ and for private servers there are projects like Snikket where the server software is developed in parallel with apps to ensure a consistent feature set.
* Screen sharing * File sharing * Sync across devices * Group chat without fuckery * Ability to delete and edit messages * Notifications of new messages
I know XMPP supports all of this and more. But find a cross platform client/server that implements all of it?
Then look at the clients for matrix, telegram, signal, teams, whatsapp, and others.
That is what Snikket is trying to be. https://github.com/snikket-im
Btw, iChat used to support all those features, but only when the other side was iChat too.
I think it has taken too long for the community to realise what an asset a good brand can be, and that it's hard to rally normal users around a protocol and an ecosystem rather than a product. I think that's growing to change though, with a number of user-focused projects around the ecosystem (including Snikket, the one I'm working on).
I only know about MAM which doesn't work with E2EE... and depends on the individual chatroom's server.
> Both XMPP and Matrix have bridges between each other and third-party protocols (proprietary and open ones), which makes any attempt to create false competition between them an unhelpful effort.
Bridges don't really work with E2EE anymore...
Soo first you want E2EE so nobody can tap into your conversations, and then you want to receive text of you messages from the server archive? :hmm emoji:
Android version is in rewrite (decided to dump existing app because of tech debt issues), but I have zero reasons to doubt that it will work (hopefully by august).
iOS version is not yet released on Appstore because we need to work out some more issues (not too many), but I can give you access to it, if you're interested.
(We're currently not considering everything ready for public release, thus apps available to public lag from what we ship to our customers and use ourselves)
However, instead of arguing I think it might be better to give a short summary about the state of XMPP from a users perpective. I use XMPP every day for several years now. In my opinion the best clients are the following:
- Linux -> Gajim (also available for Windows)
- iOS -> Siskin
- Android -> Quicksy/Conversations
Especially Siskin has made great progress over the past year. While these clients can do most of the things you would expect from a modern IM client, there are still a few weak spots:
- The combination of OMEMO (End-to-end encryption) while using multiple (clients over longer periods of time) can lead to situations where some clients can't decrypt messages. This is rarely dramatic (because one of your devices can decrypt the message), but can be annoying.
- It seems video calls across different clients are work-in-progress. Conversations to Conversations works fine, but with Siskin I had some trouble.
So the bottom line is: Most things work fine, but some things don't.
- Linux -> Psi+ (also available for Windows)
- Android -> Conversations
All FOSS.
On XMPP itself — I'm looking into it as the protocol for messaging for my federated social network. I first contemplated using Matrix, but then I looked at the spec, saw the mathematical notation for the room state synchronization algorithm, and noped outta there really fast.
* https://nlnet.nl/project/Libervia/
* https://inqlab.net/2022-01-17-activitystreams-over-xmpp.html
[1] https://blog.joinmastodon.org/2018/06/how-to-implement-a-bas...
[2] https://blog.joinmastodon.org/2018/07/how-to-make-friends-an...
I do a variation of this for ActivityPub in my project already. Except I don't implement c2s ActivityPub at all.
The chat standard XMPP (IETF): https://www.freie-messenger.de/sys_xmpp (german)
Thoughts on interoperabiliy: https://www.freie-messenger.de/en/begriffe/interoperabilitae...
Quick overview of messenger systems (DE, EN, FR, ES, NL, ...): https://www.freie-messenger.de/en/systemvergleich
Because there are so many comments comparing XMPP with Matrix: https://www.freie-messenger.de/en/systemvergleich/xmpp-matri...
The author grossly misunderstands XMPP. The first letter X stands for "eXtensible", and extensible it is.
Chat history, milti-device sync, reliable message delivery is not an afterthought, it is extension (and btw they all are solved problems as of now) The protocol is by design modular, and it is a strength, not weakness.
You can't realistically expect all the world to share the same version of your monolithic messaging protocol. Different instances will support different features, some implementations will lag on with newer features, etc. The only possible answer to this is modularity and legacy fallbacks for some features that are not supported by remote party.
Sure, the protocol could develop more rapidly. I actually believe that XMPP survives despite efforts of the XSF coincils, not thanks to them. However, the core of the protocol is strong and allows to build communication products with UX not worse than Telegram or Discord. And if you lack some crucial feature like a properly working group chat, you can make your own XMPP extension - because eXtensible it is.
Also. You don't really need any of them to send messages between servers. All this stuff matters only for communication between your clients and your own server.
XML is always painful but XMPP brings it to another level by not playing nice with most parsers which expect complete documents, I ended up having to roll my own parser.
But perhaps the situation in XML land has improved since, I sincerely hope so for the people writing today's clients.
Professionally I've been involved in one project that used XMPP as a data protocol between back end services. It was eventually phased out in favor of REST because of buggy libraries which meant capturing messages in queues and replaying whenever the thing fell over.
>Today you expect a messenger to be a database.
I do? Not all of us want the levels of resource utilization this approach requires. Why would I want to be able to edit/delete old messages? I can just send new ones.
If by "push" you mean sending notifications through 3rd party servers like google's Firebase, I'd like not to use it.
I was using XMPP on mobile for years, receiving messages in background normal way almost with no battery drain.
I also run k9mail 24/7 with my personal IMAP server, receiving messages in realtime (imap idle) with almost no battery waste.
Battery is wasted more by bloated frameworks like Electron rather than by network activity itself.
These days there's also UnifiedPush which is an open standard for push on Linux and Android. FluffyChat and SchildiChat (both Matrix clients) support it. I actually switched to SchildiChat from Element because Element's notification service was using too much RAM.
Mobile push is a combination of brain damage and bizarre propaganda targeting developers.
It's not activity itself but what app does in background in reaction to incoming packets. Clients I used (Bombus and Xabber) didn't support that XEP with no problem. App may just ignore some stanzas when not needed.
A lot of effort went into Prosody 0.12, which we released recently. It's unfortunate that mod_cloud_notify didn't quite make the cut for that version, but it was better than delaying the long overdue release even further. The vast majority of work on Prosody is unpaid volunteer time, and as with most open-source work there are certain kinds of tasks which are hard to find funding for, leading to the old "it'll be done when it's done" because we (the developers) simultaneously need to pay our bills.
Also, in recognition of many desirable things being in community modules, one thing we did prioritize in 0.12 was the plugin installer, which should make it easier for people to add additional modules to their Prosody installation.