Easy XMPP: What are we doing here?
mail.jabber.org
mail.jabber.org
* Signal users are not anonymous; Signal requires users' phone numbers.
* Signal is centralized. Is there a way to run your own Signal server?
* Signal uses Google Chrome on the desktop (and Android?) and Google Play Services (or some part of them) on Android (I don't know about iOS). Whatever you think of Google's intentions, they are one of the leading surveillance organizations in the world. Signal users must trust Google.
* Is Signal's system (not user data) fully open and transparent, end-to-end?
* Would I need to trust Signal more than I would need to trust Jabber?
----
EDIT: I came across this comment from Moxie in late 2015 which addresses some of these issues and provides a broader view:
https://news.ycombinator.com/item?id=10665520
If we were going to rank our priorities, they would be in this order:
1) Make mass surveillance impossible.
2) Stop targeted attacks against crypto nerds.
It's not that we don't find #2 laudable, but optimizing for #1 takes precedence when we're making decisions.
If you don't want to use your phone number, don't use it. You can register with any GV, Twilio, Voicepulse, or other throwaway VoIP number.
If you don't want to run Chrome, use Chromium instead.
If you don't want to use Google Play Services, use GcmCore.
The world you want this software for is not the world that everyone else lives in. You can certainly make it work in that world with a little effort, but because of how we've prioritized our objectives, that's not the default experience.
True for Signal. XMPP is using JIDs which are functionally comparable to email addresses, and similarly anonymous (or not).
Is there a way to run your own Signal server?
The Signal client is open source, the server mostly so. You could create your own semi-Signal community, but it would not be able to talk to official-Signal users.
XMPP is designed for federation by default, and you can run your own server, akin to email. Though some server operators decide to disable or cripple the federation feature (Facebook, Gmail).
Would I need to trust Signal more than I would need to trust Jabber?
This is a complicated matter. You need to trust the server operators to handle metadata in both scenarios, but you can run your own server with XMPP. You need to trust the client developer, which is Signal in one case, or the developer of your choice (or yourself) in the other. You need to trust the device manufacturer/Google in both cases.
It's possible that they grandfathered you in, if you were already federated when they put on the breaks, and your server wasn't guilty of sending spam.
Or it's possible that I'm wrong and they just got really aggressive about banning new servers if anything even remotely spammy came out of them, and it's really the spammers complaining that "Google won't let anyone new federate."
While OTR is a possible solution, it has its own can of worms (multi-device, offline use).
Besides of this, Gmail xmpp silently fails if the other party upgraded to hangouts.
[0] https://github.com/stpeter/manifesto/blob/master/manifesto.t...
The issue is that you see 3 people are online, you can also receive their messages, but they can't see anything you write[1]. This behavior is worse than outright blocking, because it creates confusion at first seems like everything works and why people are complaining, then makes you suspicious that perhaps Jabber is broken, and in order to talk to your friends you might start using gmail.com.
[1] Initially this happened to anyone who converted to hangouts. I don't think there's option anymore to opt out from hangouts and get back to GTalk.
[1] https://cloud.google.com/appengine/docs/deprecations/xmpp
Also, in some countries like the US, you can buy 'burner' phones without any need for identification--no contract, no ID.
I believe that it is not transparent in the usual sense, because the server-side component is has no source available, but also that it does not need to be transparent in a cryptographic sense: the amount of information known by the Signal servers is minimal, and doesn't include messages, contact graphs, or even which users are communicating with each other. It does include what users they have and what phone numbers, and when they log in / where from: I assume they could also learn whether someone is using Signal at a given time and how much (from bytes transferred) but nothing more interesting about the conversation. I believe the use of Google Cloud Messaging is the same way.
I haven't looked into the protocol myself and I'm not sure if all of this is true. In particular, while OWS says they don't have any "records" of who a user is communicating with (https://whispersystems.org/bigbrother/eastern-virginia-grand...), I'm not actually sure if this means they're just not logging it, and a system could be built to record this.
It's XMPP based and offers many Signal-like features. Seems very similar to the goals of this.
This is perhaps the biggest problem Free Software programs have when you start treading away from things like a browser. There is a huge hostility to ease of use and building things that are Powerful Enough. People don't care if you have a lisp scripting language embedded in your app, they care about how long it takes to get started doing the stuff they care about with your software. Until usability becomes more important, then XMPP isn't going to gain traction.
It is a shame that nothing by the likes exists in the XMPP-sphere.
EDIT: As a side note: Daniel Gultsch from the conversations-fame is doing a great job by providing/developing a realy awesome XMPP client for android and pushing the standard forward.
When it comes to protocols (as opposed to "services"), they're seldom replaced. I mean, we're still using IRC for open source projects. Matrix/Riot gives me a glimpse of the future.
I recommend reading this: http://www.titus-stahl.de/blog/2016/12/21/encrypted-messenge...
Matrix/Riot does tend to break down in channels with 30k users all chatting, sending hundredthousands of messages per minute. (Which is a real use case of IRC).
If that is a meaningful discussion is another topic, but many chats for live evenys, especially on Twitch, have these numbers of users and messages.
Other networks have similar events.
And yes, most of the problems I describe only appear on Twitch, or other livestream chats.
"enormously. much of synapse's algorithms and DB schema are still unoptimised, plus python is not exactly renowned for being super-fast. we're effectively going through rewriting chunks of synapse - e.g. the new state storage representation in 0.18, and meanwhile Ruma is going through writing a cleanroom impl in Rust that should be a bajillion times faster :)"
(https://www.reddit.com/r/linux/comments/56xzyo/matrix_is_a_n...)
But even then, Matrix will likely not scale well enough to replace Twitch's backend, or to be able to nicely handle QuakeNet's event channels.
You'll also have to have a native client at that point, because the webclient also breaks down at those amounts. And you'll need to use a binary protocol or simplistic text protocol between client and server, because otherwise the user's mobile connection won't be able to handle that traffic.
Matrix will have to change a lot of things before that scaling is easy.
For bridges that you can run on your on "homeserver" (if you indeed want to run your own server!), see https://matrix.org/docs/projects/try-matrix-now.html#applica....
If you use the matrix.org homeserver, they bridge with a number of IRC networks, including freenode and OFTC. The closest thing to an official list that I've seen is https://github.com/matrix-org/matrix-appservice-irc/issues/2....
I can only imagine an engineer could see it this way. I found it rather confusing. I tried joining #debian and I kept getting "invitation" messages from some IRC bridge.
This is a great point made later on the thread.
And already happened in some countries, for example, with whatsapp.
Social pressure was big enough to revert it in a reasonable time, but time is very relative on how much you or your business relies on it.
I'm sure that if Moxie turned evil it would be difficult for the project to survive, but that's also one of the reasons they've been successful. They have control over what they're building, and they can move fast and make breaking changes. It's very difficult to build something beautiful when you can't make breaking changes.
While when your XMPP server admin does something weird or a country breaks it you are locked in.
For Signal you use an ID which is not owned by Signal. So for Signal and alike the decentralisation doesn't come from the service but from your own social graph which is your phone book on your smartphone.
Moxie explained it much better I think. https://whispersystems.org/blog/contact-discovery/
I suspect that relying on IDs owned by another system can have its drawbacks. Maybe it's not likely that an entire country is forced to change phone numbers. It's still possible to lose a number for various reasons (most entirely benign). An oppressive government could definitely disrupt communication of suspects by forcing the phone company(-ies) to terminate their contracts and unlist their numbers.
(But IDs being piggy-backed to a different system which is not easily disrupted on a mass scale does have some advantages, too.)
I got my mobile number in 1999 and already switched twice with the number. Sadly it is kinda expensive to take the number with you, 25 to 30 euros.
If it's not free software, you have neither freedom nor privacy. If it's not decentralized or federated, you have neither freedom nor privacy. The only contenders for freedom and privacy are XMPP and Matrix. All the others are contenders for money and popularity, but not for freedom and privacy.
Popularity and money are useful and not inherently incompatible with freedom and privacy, but they are secondary. Creating a new application that is not federated or decentralized does not help, no matter how much more popular or convenient it is.
Unfortunately, for the vast majority of people, that just isn't the case. I would love it if I could use XMPP-based services to talk to everyone I care to talk to, but, in practice, I use 8 different messaging services, and all of them are walled-garden non-XMPP services.
In the end, the people I know who care about privacy and security communicate with me using Signal (can count that number on one hand), and the rest use one of the others.
You can say and wish all you want that popularity and money are secondary, but in the world we actually live in, they're #1. I expect that won't change until and unless something horrible happens and "regular people" have their privacy compromised in a way that causes them real, tangible, extreme harm.
This applies to centralised/proprietary systems as well, including those 8 walled gardens that you use.
When Slack was created, nobody was using it. When Signal came out, nobody was using it. Same with Telegram, Skype, Twitter and all the other centralised/proprietary systems.
Some of those did manage to gain enough users for positive network effects to kick in, so "nobody uses it" isn't specific to FOSS/decentralised/federated systems, it applies to messaging systems in general. The fact that so many messaging systems keep being created, and so many users hop from one to another, shows that it's not a particularly strong argument either.
The real problems for systems like XMPP are the levels of investment required in marketing, UI design, etc. to compete with the proprietary systems.
As mentioned by someone in the linked thread, part of the user problem is that users can't even conceive of a federated IM system and don't know what one might be like. Asking someone to get a XMPP client so they can communicate with you normally just ends in confusion. There is no download button on the xmpp.com site.
* https://mail.jabber.org/pipermail/standards/2017-January/031...
> Asking someone to get a XMPP client so they can
> communicate with you normally just ends in confusion.
Asking someone to used an XMPP client, ideally should be the same as asking someone to use a torrent client.
It is the same, and the majority of people can't manage to do either.
The adoption rates show a general failure of the model.
Server-side transparent interoperability like Riot has with several services, however, seems extremely promising as a way to break that cycle.
https://en.wikipedia.org/wiki/XMPP#Connecting_to_other_proto...
XMPP was a tower built too tall and wide and it crumbled under its own optiomal extension weight.
Just bridges mean you are still selling your soul to a primary operator, which prevents trust.
Federation and bridges without extension hell means you can use whatever homeserver you want, bridge to whatever you need to, and finally have working interoperable IM.
Some clients are excellent, like conversations.im. The problem is accidental complexity. Perhaps a solution would be to have a meta XEP that aggregates a few basic XEPs and defines a minimum common denominator.
Google gtalk predates facebook and happened at different times, IIRC the change to a closed chat service was a reaction due to facebook abusing the openness to scrape contacts.
Or it's part of the protocol, but design decisions lead to client interoperability problems?
Maybe there's a third answer, but both of these are problems with XMPP.
On this topic; I wouldn't mind hosting my own XMPP service, but most clients aren't good enough (specially on Android). I can't advocate for a service (protocol?) that can't offer some basics that are covered and seamless everywhere else (eg, sending files, showing media files in the client itself, etc).
I understand standards are slow, but when I think how difficult was to get avatar support everywhere... it makes me sad.
After many years, I gave up on XMPP recently for "internal family" communications. I'm using Telegram now, although it really bothers me that I need a phone to start using it.
You can't have two phones synchronizing yet, though.
And then when the NSA decided that they need their number CC'ed, your friends won't ask you...
But I guess requiring a smartphone for a privacy anything is quite far fetched too.
I'm worried that the only ones seem to be email and the web, both of which came into existence when the internet was small and academic and it was natural for universities to decentralize. And running email on your own is getting increasingly hard because of spam and IP reputation. (We seem to have more-or-less won the war on spam, but at the cost of making email much less decentralized than it used to be.)
There's a mention of BitTorrent elsewhere in these comments, and there might be an argument for Bitcoin. But even for IRC, people tend not to run their own servers (although they could); there are a very small number of IRC networks, run by random people.
I would love to see a decentralized and federated team chat app along the lines of Slack or Discord, but I'm having trouble believing that such a thing would have a chance of success in a post-1995 internet.
Are you sure ? https://mailinabox.email/
(Or, in other words, if this does work, why do people who have small- to medium-scale deliverability needs use a dedicated email-sending provider instead of just using a t2.nano running an AMI with Postfix?)
Around the mid-2000s, IM networks started getting tired of constantly changing their protocols to thwart third-party reverse engineering efforts like Microsoft logging into AIM, libpurple (Pidgin), or Trillian. But then Google Talk appeared [1] in 2006 inside the coveted invite-only Gmail, supporting XMPP, and significantly raised the bar.
So interoperability became a tool to maintain market share. The underdogs WLM and Yahoo started seamless interop [1] in July 2006, while Google Talk and AIM started a limited interop [1] in 2007. AIM briefly dabbled with XMPP it in 2008 [2] (great source -- see comments for AIM's response).
In the meantime, Facebook opened up for everyone, introduced Chat and rapidly lured away the myspace/AIM generation, becoming a major player in chat. Facebook introduced XMPP in February 2010 [3] but discontinued it [4] in 2015 after having deprecated it the year prior. This neatly coincided with their announcement to monetize the Messenger ecosystem, in ways that require a captive client [5].
Other vendors are similarly pursuing monetization within the client -- Snapchat and Kik as a content portal [6][7][8][9], Google as a context-aware assistant, Microsoft is lost at sea, Whatsapp as a Facebook data mining scheme, the Asian apps as a combination of all other techniques and microtranactions -- when anyone can bring a third-party client, their monetization strategy suffers. This makes XMPP's deployment future exceedingly bleak, perhaps restricted solely to corporate deployments.
[1] https://news.ycombinator.com/item?id=11114518 [2] https://web.archive.org/web/20080120143857/http://florianjen... [3] http://web.archive.org/web/20100318030410/http://developers.... [4] https://developers.facebook.com/docs/chat [5] https://developers.facebook.com/blog/post/2015/03/25/introdu... [6] https://news.ycombinator.com/item?id=11935956#11941090 [7] https://news.ycombinator.com/item?id=12000854#12002773 [8] https://news.ycombinator.com/item?id=12206846#12207459 [9] https://news.ycombinator.com/item?id=12272973#12273447
Nice sum up of events, though you missed that Microsoft bought Skype.
Interestingly, email didn't, despite acronyms like POP3, IMAP, SMTP being a frequent occurrence once you try to supply an email client of your choice. So what was different about email, once internet users started escaping the walled gardens of old?
That said I do believe some of our XMPP customers are starting to look around and ask for REST/Websocket based APIs as their new dev team hires look to reinvent the wheel ;)
Seriously speaking though - I think XMPP missed the boat as far as messaging goes. Smart phone apps took everyone unawares and XMPP didn't move fast enough to provide a solution for low power devices on high latency networks.
That said I wonder if there is a future in which XMPP does prove to be a compelling solution. I wouldn't put money on it, but even today a full duplex API which is highly secure, async and which offers schema enforced validation of your messages sounds pretty damn compelling. That said there is a lot of progress with things like STOMP, RabbitMQ etc. providing the pieces necessary to replace XMPP.
It's a pity since XMPP makes it so damn easy to build reliable, realtime apps.
Bonus: I once discovered I had two of the main culprits for XMPP getting one of its worst misfeatures (lack of sensible framing) in the same room so I got to yell at them for it.
XMPP was form over function. And it wasn't even pleasant form.
The alternative is to try to anticipate everything anyone would want to do with a protocol. That has problems as well. The XMPP approach is to accept everything and aggressively ignore everything not supported. XML is quite compatible with this approach.
XMPP uses the XML as the framing. An approach that could actually be considered clever if one is striving for simplicity in a protocol.
Using XML framing in XMPP was the opposite of simplicity. (Sure it was simple in the sense that it required zero experience with actual implementation of protocols to arrive at the conclusion, but the result was something that was harder to implement properly).
It is "simple" for moronic, bad, implementations of the protocol, but it only complicates the situation when you need performance, quality and efficiency. It complicates it greatly. In essence, you end up having to write two parsers: a shallow framing parser and a deep parser.
And if you think you are going to get any help from the fact that there are lots of XML parsers: I've got bad news for you. There's lots of XML parsers that are meant to parse documents. Not millions of simultaneous, "endless" streams of data from dodgy clients.
XMPP is not good protocol design. It is brutally stupid protocol design.
(An irony is that right now there are several areas where you would want to use a messaging protocol for small devices. This ought to be the moment where a messaging standard had a chance to shine. And XMPP ends up being one of the least desirable protocols because so little care was taken in designing it)
You obviously wouldn't use such a parser for XMPP. You would use a parser designed for the purpose. Very few people would ever have to even think about the issue, there are XMPP libraries available for pretty much any environment in existence.
XMPP also isn't usable for many new IoT applications where it should have had its opportunity to shine, but where there just isn't enough memory to deal with the ridiculous bulk.
But I do get that most developers don't really try all that hard. I mean, look at the resource usage of Slack. It is an application that does _nothing_ that requires CPU, yet it gobbles it up. It tells you something about the kinds of people that write chat applications today. Not exactly the sharpest tools in the shed.
Well presumably this hypothetical IoT application would have to support TCP/IP. Against that, a few hundred bytes of state table for the limited subset of XML required for basic XMPP would count for much.
What servers, and what clients implement it though? No idea...
It's still experimental, but it should replace message archiving "soon" once a few final kinks have been worked out (that being said, most servers and clients implement it already and in my mind it's ready for production).
[1]: http://www.wired.com/2015/03/slack-admits-hacked-enables-2-f...
I think the solution here would be a "lightweight" federation of 3 or 4 entities following a common charter and covering a wide geographical area. If one drops out for some reason, the others will still ensure the service remains alive.
The actual solution has been known for ages: decentralization, federation, peer-to-peer. pretty much what the internet was designed to be.
Not if the 3 entities reside in different countries/jurisdictions.
> decentralization, federation, peer-to-peer. pretty much what the internet was designed to be.
While I agree with you from the theoretical standpoint, it's been already shown that neither users nor providers care about that. A federation is not by any means easy to maintain (just look at the amount of work spent in keeping e-mail relatively free from spam and usable). So, either the federation in question is small enough that you can keep it under control or you're better off researching a fully decentralized system that is resilient to all the problems all other services have (doesn't sound easy to me).
It gets tiring to convince people to install Signal, Wire, etc.
though many people are just too lazy to replace even ugly default Messaging app, so wet just use it with my wife and parents since I boycott Messenger/Whatsapp, just in case for video calls I have Skype Mingo (it's sad that out of billion people in India it's difficult to find one person which would upload always newest build to apkmirror)
Why give away your SMS to a messaging app that purposely dropped their encryption to focus on online chat which is useless to me as I don't have a data plan?
[1]: https://whispersystems.org/blog/goodbye-encrypted-sms/ [2]: https://silence.im/
When Slack is bought by Google and merged with Hangouts, and WhatsApp changes over to a new all-messages-delivered-by-the-NSA protocol, email will still exist.
Hah, Google, have multiple messaging systems and bother to merge them?
Why can't I run my own signal server? Why doesn't the client support this?
We need something like Signal that that isn't under the control of a single party.
For some reason the people are reinventing the wheel and the problems that come with it that have already been addressed.
We actually need signal to go away and disappear for the very reason that is a walled garden casting shadows on what we actually need to grow in the long run which would be matrix and xmpp at the moment.