FastMail: Shutting down our XMPP chat service | Hacker News Reader

FastMail: Shutting down our XMPP chat service

blog.fastmail.com

186 pointskevinchen122 comments

While the FastMail shutting down XMPP is about low usage, the real cause is that Federated Chat failed to work.

Federation is something that is rarely promoted in modern ecosystems. Before Twitter/Facebook, we had RSS/Atom, but these have all failed as companies pour massives investments into "winner take all" approaches to their products. In the short term, it is easy to see why: Why compromise on iteration of features and UX by allowing a competitor to interoperate? Why wait for a new field to be standardized in Atom?

We see it all the time in the consumer space, federation has died, but even look in the infrastructure space, something like Docker: Yes, Docker Hub, is "kinda" open, I mean, anyone can create an account, but it's not a federated scheme where everyone can host their own Images easily. But we use it anyways, because its a better user experience, and its subsidized by a company not directly making revenue on it yet, but wanting to control the user experience.

So we drop support for any federated user experience pretty quickly. And now we are all on Slack, on Facebook, on Twitter.... And it is mostly great. Or we wouldn't still be there.

But I still wish for a federated future, but I doubt it will happen.

It's also instructive to reflect over the other federated internet communications protocol: E-mail.

It pretty much works, and that's great, but barely a week goes by when HN doesn't have an article lamenting the lack of useful spam-control or encryption/sender verification and many other things. E-mail is still a very poor medium to facilitate long threads with many participants. E-mail as notifications (push) is lacking. E-mail as an interface to software systems (e.g. ticket tracking systems) is lacking.

These deficiencies are all easy to fix in a technical sense, but practically impossible to address due to the federated nature of the system. The few changes we do end up getting (HTML bodies, and even MIME before then) are long and painful, and are largely resolved by "might is right". Others unilaterally break fundamental features, such as curbing spam by essentially banning decentralised SMTP. If you rely on being able to send out large quantities of email, you either have to pay a gatekeeper, or hire a specialist in the dark arts of actually have your email go through. This dark art consists of keeping track of the many ways in which the large players in the e-mail world systematically break the protocol to combat spam.

We have essentially abandoned the federated nature of email and handed over one-sided control to a small group of companies (Google, Microsoft, Yahoo, perhaps a few other) in order to effectively curb spam. The alternative would have been the end of email.

XMPP fails because it tries to, indeed needs to, boil the ocean. There's an extension, or two or seven, for every conceivable feature and use case, and so you need to choose which set of features in a n-furcated (where n is tragically large) set you want/need, and live with the fact the these aren't perfectly compatible with deployments that has made other choices. And so the federation and standards compatibility is really only theoretical, and then, what's the point? If spam killed truly federated email (and the need for actual working encryption might well finish it off), mobile killed XMPP.

Hopefully, at some point, the IM features of Slack, Twitter, Google Hangouts, iMessage, HipChat, Yammer, Skype, Whatsapp and friends are going to converge enough that an open source application can emerge and define a basic set of features through domination, in much the same way Wordpress ate the blogging world, and that can then lay the foundation for interoperability.

> E-mail as notifications (push) is lacking

Let me introduce you to IMAP IDLE.

> E-mail as an interface to software systems (e.g. ticket tracking systems) is lacking.

Jira, for example, has a pretty good one.

> We have essentially abandoned the federated nature of email and handed over one-sided control to a small group of companies (Google, Microsoft, Yahoo, perhaps a few other) in order to effectively curb spam. The alternative would have been the end of email.

I've been running my own mail stack for 8+ years. The reason is simple: only I can mess it up. No 3rd party to lock me out from my communication, no 3rd party to decide what I can send. If it fails, it's my fault only. And I'm still using email. I do occasionally get spam, but most of them is already dropped by blacklisting and a few intelligent rule; the rest is catched by dspam.

XMPP is overcomplicated, solving nothing IRC can't solve already ( imho, of course ); that is why it's failing.

What we need for the more open source approach is people to value their communication and not to trust 3rd party because the look/act cool.

> Hopefully, at some point, the IM features of Slack, Twitter, Google Hangouts, iMessage, HipChat, Yammer, Skype, Whatsapp and friends are going to converge enough that an open source application can emerge and define a basic set of features through domination, in much the same way Wordpress ate the blogging world, and that can then lay the foundation for interoperability.

They won't. WordPress was open from the start and the current trends are locking-in APIs, not open standards. There is no profit in openness.

> no 3rd party to decide what I can send

It's my understanding that the biggest hurdle in owning your own email stack is that a lot of 3rd parties decide what they'll receive, and hold senders to very strict standards. Have you experienced many issues with recipients on various platforms not receiving your emails?

I have worked in this area for a long time, and it's very rare that this causes any problems unless you are a spammer (or "news-letter sender"). You won't end up on black lists unless there is a mistake. It's an annoyance when that happens, but that happens for the major players as well.

I often feel the ambivalence to your own mail servers is unfounded. A basic apt-get for Postfix and Dovecot will give you secure defaults. The only thing you need to do then is to set up the certificates, the dns records, and keep up with what happens in the future. It's absolutely comparable to running your own web server, where you also need to change a cert or two over time.

Receiving was never an issue; landing in spam, is sent from some exotic software, that happened.

But I may have chosen the words; the main reasons it that I don't want anyone to let decide or mistakenly delete/close/block me out of my own account.

> They won't. WordPress was open from the start and the current trends are locking-in APIs, not open standards. There is no profit in openness.

They (the incumbents) won't. But someone will, in the same way that WordPress replaced hosted blog platforms for many. Might be wishful thinking, but this seems like it could happen some day.

RSS/Atom luckily is not dead. Barely anyone uses it because most people don't know it exists and/or what it does.

However, there's barely a blog out there that doesn't support feeds and I heavily use them in Thunderbird without a problem.

Regarding Docker: You can easily run your own Docker container registry. The DockerHub UI isn't open source, but the underlying server and protocol for fetching/uploading images is.

See: https://hub.docker.com/_/registry/ https://github.com/docker/distribution https://blog.docker.com/2013/07/how-to-use-your-own-registry...

There are several companies that have implemented alternative container registries. Google (Google Container Registry), CoreOS (Quay) , and JFrog (Artifactory) all come to mind.

> the real cause is that Federated Chat failed to work.

Mostly because some industry giants first moved to support it then yanked the rug out from under it when they realized it reduced the amount of end-user lock-in.

Also: IRC.

RSS is not federated. Each server serves its own content, and clients connect directly to them.

It's decentralized, not federated, just like the web.

I think RSS failed because of the bad user experience it provides. Unless you are following hundreds of feeders, it's much easier to just go to their site, than to open a reader, get maybe one line without context - or some text that isn't in the article at all, click there and go to the site.

It's funny, RSS is in principle the same as many socially-powered link aggregators (reddit, HN, 9gag, etc) except you're fully in control of the content you subscribe to. I guess that introduces the hurdle of finding content worth subscribing to, but most online RSS aggregators I've seen have offered some sort of discoverability systems.

There's lots of misery and soul searching about the failure of federated chat on here, but all the slack-likes that are gaining usage are proof that the opportunity to fix this is still open.

A federated chat that had a good user experience and ideally a feature that appeals to a niche (e.g. easy voice chat for gamers like discord app) could still make headway.

In fact, the proliferation of new niche clients with relatively easy integrations is the kind of situation in which a federation standard could thrive.

"we had RSS/Atom, but these have all failed as companies pour massives investments into "winner take all" approaches to their products"

One area which hasn't (yet?) succumbed to this is podcasts. The podcast clients I use (Overcast on iOS and Pocket Casts on Android) will accept an RSS URL, and most (all?) podcasts provide such a URL.

I think that's because iTunes uses RSS/Atom for podcasts under the hood. If you want your podcast to show up on iTunes, you simply must have RSS.

My issue with that is the first great online walled garden was AOL and they were competing against (using and cutting off) SMTP. So there was a ready made, open, federated solution waiting to burst out.

Whereas in Facebook, Google etc, the open solution is usually following the walled garden. I yearn for open solutions but they will have to follow open internet access (local grids) and then the economics of sending a packet to SF in order to message my next door neighbour becomes more obvious.

What a sad day.

A hearty Fuck You to Google, Facebook, and Apple for tearing apart this open federated standard. Users don't benefit from being forced to use your shitty walled-garden proprietary communication apps.

(Yes, forced; if I want to talk to my friends, it usually has to be via whatever proprietary chat network is in vogue at the moment. There was a brief golden age of IM during which GChat (federated XMPP) was king, but GChat's been in its death throes for years, and the current flavors-of-the-week are AIM and MSN all over again.)

XMPP lacks a fundamental feature: it does not echo your messages back to your other devices. This made it uniquely unsuitable for the mobile world: if you start a conversation with someone on your computer, and then move to your phone, all you see on the phone are the other person's messages, as if they had been talking to themselves. (Yes, someone did eventually write an extension for that, which nobody ended up supporting.)

I can't even say that XMPP failed to adapt to mobile. It's more that XMPP was a terrible protocol with a gigantic flaw that should not have been there in the first place, and mobile just made the flaw obvious to everyone.

All these problems have been solved in XMPP years ago, and there are some clients fully supporting the mobile experience and usage of multiple clients.

On the Desktop, Swift and Gajim come to my mind, on Android Conversations is getting really awesome (and yaxim, which I am the author of, supports the basic multi-client/mobile use case you are interested in).

That said, the XMPP protocol is not perfect, and it is lacking some important elements for a 2010+ mobile chat developer, but considering that it was initially developed in 1999, that mobile chat is only one of its many use cases, and that there are extensions covering all you have asked for, the critique is not appropriate.

[0] http://swift.im/swift.html

[1] http://gajim.org/

[2] http://conversations.im/

[3] https://yaxim.org/

That's exactly the problem, though. There are so many extensions that it's about as much work finding out which ones are even needed and how they interact for your case (in this case the replay) as is implementing, say, a good chunk of IRC.

And no, I'm not exaggerating, I've done both and decided that writing XMPP clients is not for me a long time ago.

Addendum: I really love XMPP, but I do think the many many extensions have hurt it more than one defined spec. And yes, I know it supports many use cases, but I simply only care for 1:1 and multi user chat.

You are right. The XMPP Standards Foundation is currently preparing to create a list of "must-have" client and server XEPs for certain use cases, like Instant Messaging or IoT. This will hopefully make the path easier for future application developers.

Just for the record, we are also working on a next-gen XMPP client, with decentralized blogging, file sharing, end 2 end encryption and all, and planing to do an Android port. Other projects like Movim also do similar things too, XMPP is definitely very alive these days.

Sending message to other devices is managed by message carbons which is in last call for being a standard XEP, and support is becoming more and more common.

XMPP is all but deprecated, very active, working well, with a large ecosystem. But is really badly known by most of people(I have by the way started a series of article to explain it: http://www.goffi.org/tag/talk_xmpp).

It's really difficult to be heard these days, but there is a new wave in XMPP world (check our project: http://salut-a-toi.org or movim: http://movim.eu).

XMPP is all but deprecated...

Just a short English usage note: "all but" means something like "almost". Judging by the rest of your post, you probably mean "XMPP is far from deprecated" here.

yes indeed, I'm not a native speaker as you can guess.

Thanks for the correction :)

I usually read "X is all but Y" as "X is Y in all but name", but there are subtle distinctions that could be hard for a non-native speakers[1] to grasp without knowing about them.

1: http://english.stackexchange.com/questions/9967/all-but-idio...

Is there any way to handle message histories that survive the XMPP server (ejabberd) being restarted?

At the moment all the MUC room histories vanish if I need to restart ejabberd which is a farce in a cloud world...

And message carbons work, but since ejabberd only seems to support a single auth token meaning that if users connect with multiple devices they can't actually log out of only a single one without pulling the plug on all connected devices.

Are these only ejabberd issues or issues in the XMPP specs?

BTW this is for realtime chat on for an app, nothing federated... I'm open to non-XMPP options too.

> Is there any way to handle message histories that survive the XMPP server (ejabberd) being restarted?

Message Archive Management (XEP-0313) is supposed to do that.

> And message carbons work, but since ejabberd only seems to support a single auth token meaning that if users connect with multiple devices they can't actually log out of only a single one without pulling the plug on all connected devices.

Not sure to understand: if you disconnect a device, all the other devices with the same jid are disconnected? If so, it's definitely not normal, and not a spec issue. I'm also surprised to see ejabberd having this behavior, you should talk to the dev team.

> Message Archive Management (XEP-0313) is supposed to do that.

Thanks, I'll check that out.

> if you disconnect a device, all the other devices with the same jid are disconnected?

No, not on disconnection - I can't seem to find a way to tie multiple auth tokens to a single user (one per device) to allow them to log out on different devices individually. At the moment logging out one device would disconnect all clients.

My understanding is that it is not the clients that are the problem but rather there is patchy support in open source XMPP servers to utilize the new mobile friendly features.

[1] - https://www.process-one.net/en/ejabberd/protocols/

[2] - https://prosody.im/doc/xeplist

[3] - https://github.com/esl/MongooseIM#features-and-supported-sta...

Not true, XMPP does have that feature and there are multiple clients which implement this. It works amazingly well. It's not an extension, but part of the standard.

Example:

http://conversations.im/

Doesn't excuse Google, Facebook, WhatsApp, and Apple, from building closed protocols and actively thwarting attempts to make third-party apps.

(Yes, I know they're for-profit companies who have every reason not to want to relinquish any control over their network. Doesn't mean I can't hate on them for not even pretending to support consumer choice.)

> not even pretending to support consumer choice

By my way of thinking, your choice of language (not because of you, the person, but of how the words have become ingrained) is indicative of the problem: "consumer" versus "customer." We consume what the various companies choose to offer to us in the way they offer it. We are not treated as customers, valued or otherwise, in a competitive landscape. There exists a one-way flow, from the company doing the advertising to our eyeballs to consume it.

Look at the businesses people claim to like: local shops, service providers like Fastmail, and the like. In those cases the people giving them money are treated as customers who have other choices and who have made the voluntary choice to continue to associate with that business.

That's what is missing. Customer instead of consumer.

When someone raises this concern, I ask who hosts their personal email. 30-50% use a legacy free Google Apps account or Gmail. When I ask why, it's a variation on "because FastMail costs ~$40/year."

Of the rest, more than half host their own MX. While that's totally reasonable, it means the percentage of people willing to pay just $3/month to be a customer is even smaller than it seems. And $40/year is the cheapest that one could ever hope "being a customer" would cost.

So, lots of people say they want to be customers, but even when doing so is close to free, very few actually do.

(Nothing wrong with using Google because it's free, only while concurrently claiming to want to be a customer. I agree with your point and I'm a happy FastMail customer. I'm amazed FastMail can make a profit at $40 and it's a huge credit to them that they can.)

I wonder, if we were proper customers of Facebook and paid them to transport our messages, whether it would open the possibility of a Ma Bell-style antitrust lawsuit. Force interconnectivity to allow the little guys to play too.

WhatsApp is paid (after a year), but it doesn't help with supporting any open IM standard.

Yeah, I've never understood this. I've seen it now-and-then when installing WA for family members, but I (and they) have never, ever had to pay for it after using it for multiple years.

It's free since Facebook bought them. I've had Android friends mention payment screens are still on their WhatsApp though.

It was free before Facebook brought them too. It just nagged people to pay once a year, but if one didn't pay, it just kept working.

s/Customer/Citizen/ and you've hit on a large portion of the problem with representative democracy.

What about Comcast? What about airlines?

While it's not like xmpp/jabber protocol is perfect, it leaves lots of space for extensions. Any company serious about it could either implement (post 2011) or propose (pre 2011) extension like XEP-0280 (message carbon copy). There's another one for accessing archives, googlable if anyone wants it.

> It's more that XMPP was a terrible protocol with a gigantic flaw that should not have been there in the first place...

What is that flaw? Is it the flaw that you mentioned in your opening sentence?

Edit: For the downvoters: This is a serious question. I don't ask non-serious questions. :)

I think the flaw was mentioned in the first sentence of the parent's response: "XMPP lacks a fundamental feature: it does not echo your messages back to your other devices." This capability was indeed one of the major selling points of Apple's messaging system in the most recent iOS & OS X releases. If XMPP cannot keep track of multiple clients then that is a huge flaw in todays mobile world.

Unfortunately it is not true. XEP-0280 [0] (Message Carbons; since 2010) forwards your conversation to all your online clients, and XEP-0313 [1] (Message Archive Management; since 2012, but only recently implemented) allows a client to "catch up" with the history when it goes online.

[0] http://xmpp.org/extensions/xep-0280.html

[1] http://xmpp.org/extensions/xep-0313.html

Whatsapp never supported this "fundamental feature" and is globally the most used messenger in "todays mobile world" by a wide margin.

Whatsapp is built on XMPP I believe. :)

A closed proprietary non-interoperable extension on XMPP.

> I think the flaw was mentioned in the first sentence of the parent's response...

We can speculate all we want, but this just leads to confusion. :)

I blame Apple[1]. Google's chat was free, but what happens when you have a free and open protocol, and your competitors don't? They consume your chats, but you can't consume theirs. Soon, your chat software can't compete. Apple has repeatedly chosen the path or proprietary when they could do open. They are Microsoft from the 90's with a better PR department.

1: I have little evidence beyond what looks obvious, but that could definitely be wrong. I'm open to new information proving me wrong.

In 2010 Jobs promised "We're going to the standards bodies, starting tomorrow, and we're going to make FaceTime an open industry standard."

Afaik it didn't happen because of patent issues.

Patents prevent you from documenting, what is already documented in the patents ?!? I do however recognize the usage of the open standard would be limited by this. But one has options: other users could get a license, or one could try to change the protocol, to be able to avoid (some) patents, allowing both the patented and non patented way.

To me it just feels like they just don't care enough.

Party A may have an agreement or indemnification clause from patent holder B, but they may not have the right to extend that to other parties. So when C comes along and implements party A's new thing, they leave themselves open to lawsuits from party B.

Also, patents do not have to document everything. You can leave out some trade secrets, as long as the patent itself documents enough to prove it's a novel invention.

Apple has supported XMPP since back when Messages.app was still called iChat. The XMPP support is not great, like most OS built-ins, but it works. I'm still using it to chat on Facebook. If Google hadn't dropped support for XMPP, I could chat with Google users too. I fail to see how Apple is to blame.

This is exactly what I'm talking about. Apple supports XMPP, in that they read it, but they don't publish to it. Google publishing to XMPP allowed Apple users to read gchat, but nobody else could read iMessage. This provided an advantage to Apple at cost to Google (why use gchat when iMessage lets you do both?) The result? Google proprietarizes gchat so they can compete.

I'm surprised Facebook Chat still works for you? My understanding (and others' experience, from a quick Google) is that XMPP Chat has been disabled since earlier this year: https://developers.facebook.com/docs/chat They only supported exactly as long as they needed to to gain market share.

And Apple gets pretty much zero kudos from me for client XMPP support. iMessage is a proprietary walled garden; client-only XMPP support is simply their one-way valve to get people into their network and only ever had any worth because Facebook and Google used to support XMPP.

I was surprised too, but it works, minus offline messages. (I am using it right now). Do you really think Facebook ever needed the market share from XMPP? I've always assumed that Apple and Facebook only support XMPP because they use it internally (engineers being engineers...)

> And Apple gets pretty much zero kudos from me for client XMPP support.

I guess we have different expectations. Apple is a company that sells me client hardware/software. I use Mail to connect to standard IMAP servers, I use Messages to connect to a standard XMPP server and Facebook. You can apparently even buy Apple's XMPP server through the Server app on the App Store: https://en.wikipedia.org/wiki/Messages_Server The one thing they don't offer is a single, centralised XMPP server. That's not really an active attack on the standard, though.

Anyway, I live in a completely different filter bubble than you and your sibling poster - XMPP never had any momentum at all around me, so I can't really speculate on how it died.

Embrace, extend, extinguish.

Its sad to see the world of text/instant messaging devolving back to the 90s. It felt like the future was so bright not that long ago. I've basically stopped using instant messages all together and switched to an even more proprietary/archaic system (SMS). It is the only thing left that can communicate with everybody.

Of all the systems to transmit messages, SMS is probably the least proprietary.

Do you even need to ask your recipient what carrier they're using or do you just punch in their number and text away? Does it work regardless of what brand of mobile phone they have? Does it work even if their phone is temporarily disconnected from the network?

> Do you even need to ask your recipient what carrier they're using or do you just punch in their number and text away? Does it work regardless of what brand of mobile phone they have? Does it work even if their phone is temporarily disconnected from the network?

WeChat has all of those features, but I don't think anyone would regard it as other than proprietary...

The SMS protocols are indeed not proprietary (all the specs are online) but you'll either need a contract with an aggregator or with every mobile network operator you'd like to send traffic to. Not very handy. :)

I use Hangouts, but that's mainly because it integrates Google Voice, which I've had since it was Grand Central and even then, I mostly do SMS with it (including desktop).

It's actually really sad to see these events... one of my favorite things in the late 90's and early 00's was actually the voice chat rooms in Yahoo messenger, but the bots ruined that platform... it was proprietary, but it was fun.

These days, it's pretty much SMS only for me... I don't need an app that only works on my phone, not my desktop, and actively spies on me, and sucks my battery life. I do have skype which I use sometimes, mostly for those that I talk to overseas on rare occassion.

I had high hopes that XMPP or a decent replacement would evolve that actually worked well... same goes for Diaspora and a bunch of other things though. In the end people like their walled gardens. :-(

>I had high hopes that XMPP or a decent replacement would evolve that actually worked well

Matrix (http://matrix.org/) is getting there. It does work well, though it's still in early (and active) development.

> I've basically stopped using instant messages all together and switched to an even more proprietary/archaic system (SMS)

You and me both. I don't have space on my phone for every individual messaging app that my various friends use.

"Have you got whatsapp?" No "Do you use Kik?" No etc.

> [SMS] is the only thing left that can communicate with everybody.

Except for when it does not. More often than not carriers don't return the correct status codes, marking messages as delivered when in fact the recipient does not even have SMS enabled in their contract. This gets even worse with international messages that can be routed through multiple carriers' networks until they reach (or don't) their final destination.

Of course users don't benefit. But the operators of the walled-gardens do! I can't even understand how federated XMPP had any support from the large players in the first place - it's absolutely against their economical interests.

Let's be honest - we live in times when "federated" email communication would also be abandoned (with Facebook becoming the de facto platform for personal communication) if not the business users.

I wouldn't be surprised if we see embrace-extend-extinguish-style tactics from Google around business IMAP/SMTP within the next decade, if Google Apps for Business is able to take enough market share from Outlook.

Don't they already? Gmail was the embrace-extend phase. Now we are into extinguish. Try setting up your own email server, it will be marked as spam by most major webmail providers.

I use my own server, and I can't say that everyone is marking me as a spam. With correct setup and a little bit of good history gmail works fine for me. I removed SPAM label from my mail in Gmail and asked few friends to do the same. Now my mail isn't marked as spam in gmail. And for yandex.ru and mail.ru it worked out of the box.

I don't think Gmail would be able to block "own" email servers, because how would you tell them apart from company servers or web sites that send out confirmation e-mails or "forgot password" e-mails?

They do however require your server to be properly set up, with a correct reverse DNS etc. And Hotmail/Live mail does the same.

They still play good with the other few large email services. But with every year using google apps for business is becoming more and more obvious choice from economical POV. In fact I've myself recommended quite a few companies to "just use Google apps".

I guess it just takes time too switch phases.

Its our own fault for embracing those shitty walled-garden apps.

But honestly - they're not trying to appeal to us. Every time I get stressed at the Googlemonster or Facecrook I remember that all I have to do is press delete and use something else.

edit: these apps are built for 1. people who don't / didn't care in the first place 2. noobs 3. my parents 4. advertisers wallets

It doesn't matter what we do. It's the network that counts, and Google, Facebook, Apple, and Twitter know this. They have my friends, and they don't let me communicate with them unless I use their apps with an identity under their domain.

At least text messaging is still federated with portable identities. Took an act of the Feds to ensure that…

Heh, Trillian and Pidgin worked pretty well before the newest batch of walled gardens. Whatever happened to reverse engineering for interoperability?

Permanent user bans: http://androidcommunity.com/whatsapp-issues-permanent-ban-on...

I you can hack other's people accounts, it seems a good opportunity to force people out of WhatsApp. Just hack a guy's phone and connect him to WhatsApp through a third party client, get him permanently banned, and voilà!

Actually we see attempts to extend libpurple with many recent protocols. For instance telegram works on pidgin AFAIK, and I could recently see se a Tox plugin.

Clearly we cannot play a fair game with the closed whatsapp, but hacking cannot be stopped or forbidden.

After all, it was always like that. If you go back of 10 years al "regular people" had MSN and similar stuff, and we were IRCing. As a hacker, you migth feel sad, and think openness lost the war. IMHO the only difference is that nowadays we have much more "regular people" than before. They were simply sms-ing before they could access whatsapp.

I had a similar experience. I ran my own XMPP server for several years, so that my email address also worked as an XMPP/Jabber ID. I talked to people across several different services, including an increasing number of folks on Google Talk.

Then Google Talk stopped federating properly. Only a bit at first: things like presence notification didn't quite work right, and adding people didn't quite work right. And in the meantime, Hangouts popped up, and offered a more seamless audio/video experience too. Most people I know stopped using Google Talk entirely in favor of Hangouts; the few that remained used it only while the gmail web interface more-or-less supported it. Eventually, federation with Google Talk quite deliberately stopped working.

It didn't take long after that before almost everyone else I talked to on it stopped showing up, even those who didn't use Google Talk. People had "last seen" times in the months, and if I wanted to talk to those people in realtime, I sent them text messages, or Twitter DMs, or Hangout calls.

So when I upgraded my server a couple of months ago, I just didn't bother setting up the XMPP server again. Not because it didn't work, but just because it had nobody to talk to.

  > So when I upgraded my server a couple of months ago, 
  > I just didn't bother setting up the XMPP server again.
  > Not because it didn't work, but just because it had nobody
  > to talk to.

Exactly mirrors my experience. ;_;

I really enjoyed the interface of GoogleTalk, especially on Windows: http://www.snapfiles.com/screenfiles/GoogleTalk.gif

It was very minimalistic compared to MSN Messenger, sadly Google did seem to favor the web version of it, and it received no updates.

Sounds like how i watched a local IRC channel wither and die once PCs with Windows XP started shipping. This because XP came with a aggressive marketing for MSN Messenger.

Its our fault for letting our parents and cousins and friends start using imessage / hangouts / etc instead of XMPP. We are the ones to blame for its death, because the unwashed masses got locked in to regressive proprietary chat platforms over the past five years and we never pushed the issue.

But it doesn't help that XMPP was always awful. It has OpenGL syndrome - too many extensions, described too vaguely, and implemented in too many broken ways to work amongst one another. You get a swarm of mismatched servers and features, and if it did the same thing that would have saved commoner OpenGL adoption, it would probably still be king.

That problem is a blessed default implementation. If the foundations that wrote the specs for these low level pipeline APIs were also burdened to make sure the blessed implementation supported the entire standard perfectly, there would be an out of the box open source permissively licensed base for everyone else to build off, rather than the vague wording of technical documentation.

I'm just hoping Matrix becomes something. In the same way Vulkan can hopefully save universal graphics programming, Matrix might save IM, but we need to support it. I'm working on the Telepathy integration for it myself on weekends in a sketch repo.

I worked hard to get my family all on XMPP and succeeded. The problem is, they all want to be on iMessage as well, because of course that's what everyone else in the world uses. Eventually iMessage became the de-facto default even inside my family, and there was just no reason for XMPP to exist.

I think the comparison with OpenGL is stretched. The problem with OpenGL is that it was made to a describe a limited hardware pipeline corresponding to a graphics card from the mid 90s. Manipulating this state machine is cumbersome and made more difficult by the lack of easy debugging.

>I'm working on the Telepathy integration for it myself on weekends in a sketch repo.

I'm super interested in this - do you have the repo posted anywhere?

I love Telepathy and its native clients, but it doesn't support things like remote history retrieval... I feel like that would have to be done before a Matrix CM could work really well.

I feel that http://matrix.org/ Is the proper open source replacement for XMPP. I changed over to it from XMPP and haven't looked back.

Ah push notifications the feature we are all looking for in XMPP which is there (http://xmpp.org/extensions/xep-0357.html) but not implemented in any of the open source servers.

It's missing websockets support though which would be nice.

I found a nice overview of what Matrix.org is doing [2].

[1] - http://matrix.org/docs/spec/#id33

[2] - https://bloggeek.me/matrix-webrtc-interview/

Previous discussion about the death of XMPP: https://news.ycombinator.com/item?id=10483936

My comment from that thread:

> I've been running my own XMPP server for years, with federation enabled. A few years ago, it seemed like the logical successor to AIM and MSN and all those other walled garden IM systems. And how easy! My XMPP "name" was the same as my email address. One less thing to put on my business card.

> But since then, I have realised a big problem with it - no-one uses it! Today I communicate with the world by iMessage, SMS, Twitter, and email. "Instant messaging" just seems to have died as a concept entirely, replaced by yet more walled gardens like Snapchat.

> My XMPP server is being turned off for good, next week.

(It has since been turned off.)

My company's dev team is on Slack, so I log in there whenever I need to talk to them--unless we're doing a video conference, at which point we switch to Google Hangouts. Most of our clients prefer the phone, email or SMS/iMessage, but the ones that don't use Skype. If I'm chatting with friends it's probably on Facebook, unless it's on Hangouts. Any video chat my family does is over Facetime.

These services are never going to inter-operate, and I hate it. Imagine if we had to maintain different telephone numbers or different email accounts just to communicate with different groups of people. It's almost unimaginable, but that's what we have now with text and video chat.

This is core communications infrastructure. These segregated private networks have begun to displace the public telephone network in terms of popularity and importance. These are also the platforms over which video communication is being rolled out--not the public telephone network, as most people would have thought twenty-five years ago.

The FCC needs to step in and set some inter-operability rules for the major players here. I have mixed emotions suggesting this, considering that I'm generally opposed to regulation of the Internet. But unless the government steps in, can the situation ever change? Doesn't it need to?

Perhaps we should just wait for one to become a solid monopoly and then our anti-trust laws can take care of it. That seems to be the good middle-ground between freedom from regulation and freedom from monopolies. And it worked out pretty well for our phones.

> Doesn't it need to?

No. That's applying a personal preference - one that I also have - to the rest of the population. Most, and quite possibly almost all, users have chosen features above interoperability.

Yes, the situation could change, but no, it doesn't need to.

Contrast with, say, mobile phone number portability in the US. While one can still argue for or against requiring it, users obviously actually cared and used it. Even when XMPP was popular, very few users left Google.

As one of those few hundred people who actively use FastMail's XMPP, I guess I'm now in the market for an alternative. I need exactly one federated account on my domain, so a standalone service probably isn't going to make sense financially.

From a DIY perspective, what are the options? Are Ignite, Prosody, and ejabberd the only viable options? Are they all being maintained? Is there anything out there that works at a small scale; like a Node server that could run on OpenShift, or something else super lightweight?

We had a bad experience with ejabberd pre-release version 3 (had an intern spend the entire summer on it, and I put some time in too):

http://blog.fastmail.com/2012/09/26/one-step-forward-two-ste...

Honestly Prosody looks the best. We were planning to switch to it when async login support and scalability to thousands of domains was stable... but that was 2 years ago and it never quite materialised - and now DJabberd is so far behind in support for "standard XMPP" and the interoperability of the remaining servers that we just had to give up. I argued strongly against it, having spent so many bloody years on making this thing work - including tons of patches to DJabberd and some EJabberd hacking as well - but the numbers didn't lie.

Better to spend our time on exciting things like the currently-in-testing cross-domain support for Cyrus IMAPd and reverse ACL lookup that will make the LIST command blindingly fast even on big servers. I'm hoping to be able to say two steps forward, one step back this time instead :)

Did you look into MongooseIM?

Duckduckgo has their own XMPP server you can signup for, its what I use for everything. Not sure what extensions it supports (or lack thereof) since I only use it for single user chats.

Sadly DDG doesn't seem to support using my own identity via SRV records like FastMail does.

Like brogondwana says, from the commit log on their github repo, it's clear (years after that fastmail blog post) that Process One has abandoned ejabberd 3.x. ejabberd 2.x appears to continue to have updates. [0]

Having said that, for my single-account, single-site needs, ejabberd worked just fine for me for the several years between the time that I learned about the fact that Google Talk federated with XMPP servers and the time that Google stopped federating with other XMPP server operators.

I can't speak to how comprehensive ejabberd is, but for one-on-one chats, and multi-user chats, it worked just fine when I was using it.

[0] https://github.com/processone/ejabberd

> and the time that Google stopped federating with other XMPP server operators.

I see this keep getting repeated, but AFAIK there was only like a month long time they temporarily suspended federation for adding new contacts due to a spam issue? I've been talking to people on Google Talk from my (FastMail, sadly) XMPP account for the past year without issue? Am I going crazy?

I remember that roughly around the time Google declared that they were deactivating XMPP federation, I was suddenly unable to make -IIRC- s2s connections between Google Talk and my XMPP server, and message delivery failed 100% of the time. This situation persisted for quite a while (weeks? months?), and I -having noone to talk to- shut down my server.

Google might have changed their mind and quietly started federating again. I haven't checked in ages. Guess I have another project to complete this weekend. :)

I used Prosody. It's pretty simple to setup. You can get free certs from StartSSL to make it secure.

I shut off my server just recently, because no-one uses XMPP anymore.

I never tried it, since so far i am using xmpp as a gateway to some service, but sovereign has something related to xmpp - you might want to check it out - https://github.com/sovereign/sovereign

I ran prosody for a while. I found it lightweight, fairly easy to configure, and it worked quite well. Just don't forget to set up the magic srv records if you go the self host route.

In my opinion, and as a customer of Fastmail, I think this is the right call. xmpp as a usable federated chat protocol is pretty much dead.

For those who wish to keep using it, setting up your own server (prosody is pretty good!) isn't too hard these days.

We've turned chat interoperability into a business over at sameroom.io.

What other option is there, really? The other option is no interoperability.

A very relevant post: https://sameroom.io/blog/announcing-support-for-google-hango...

I can very much recommend this! Recently discovered it, and started gathering all my chats into Slack (you can of course do it in others). Especially getting Freenode and other IRC servers at the tip of my fingers in just one client, is a huge deal.

Of course, I could also just connect to everything through IRC (at least for Slack<->IRC), but the Slack client is a lot nicer on mobile than any IRC client I know of.

So how this works with federation? I don't see any open/common API.

It's "federation-as-a-service", basically.

A common use case is federation between two Slack teams (share a channel).

Another common use case is federation between a group in Skype and a channel on Slack.

Or IRC and Telegram, etc.

Making existing systems work together really well is priority one. API only makes sense once that's done.

Chrome complains about your TLS:

The certificate chain for this website contains at least one certificate that was signed using a deprecated signature algorithm based on SHA-1.

What's weird is that Qualys SSL Labs is giving you "A"s, so maybe they don't check for this condition?

This is disappointing, but I don't really blame them. Thankfully, IMAP is too established to succumb to embrace, extend, extinguish although Google seems to be trying. Chat protocols haven't been so lucky.

Hopefully federated chat will pick up, but it seems business models around social networking pretty much preclude any kind of federated communication between services. If adoption rates for Diaspora, Friendica, Pump.io, GNUSocial etc. are any indication, XMPP looks doomed. :(

Apple uses XMPP internally as a way to securely communicate.

Doesn't really change much; just thought that'd be an interesting thing to throw out.

What exactly do you mean by that? I'm really curious.

Interestingly enough, more and more of my friends and family are registering with and using my XMPP server. Jitsi on the laptops and desktops and ChatSecure on the phones are generally the clients they're using.

Two insightful previous discussions about XMPP:

https://news.ycombinator.com/item?id=10280155

https://news.ycombinator.com/item?id=10486541

I'd highly recommend people read through them, especially if you're interested in fixing the issue.

Ah FFS. It's sad to see that XMPP has come to this. We had the promise of an open Internet and it's being consciously destroyed by the big players, one platform at a time.

(By which I mean Google, Facebook, Atlassian, Slack, etc.; all of those companies who have chosen to put a nail in the coffin of the open Internet).

Oh well, guess I won't be doing online chat any more.

I wonder whether the children even believed the last generation of Roman Britons when they spoke of writing, under floor heating, goods from half a world away...

Can't speak to the others, but Atlassian shouldn't be on that list. They offer XMPP, I've used it, and it works well.

Federated XMPP? Or is it just that you can hook an XMPP client up to their walled garden?

Well I hate to see any features shut down even if the alternative means "unsupported operation"...

Opera has been doing that a -lot-, the biggest cases being Unite and soon Link.

One thing though:

>Meanwhile, our XMPP service is now somewhat behind the cutting edge, and lacks support for many of the recent XMPP extensions that the dedicated XMPP users are now beginning to request.

Does that mean if they kept silent the service would just run somewhere in the corner, forgotten? ^^

All I say want to say is we need open protocols (and standards), period. We need choice. Those who want to create wall-gardens are free to do so but we need open protocols and educate users and developers about how important they are. RSS, XMPP and others are used everyday, even by corporations so eager to user proprietary ones for their to keep their users prisoners.

To all of those moaning about how great XMPP was: It wasn't.

In a multi-device/mobile world XMPP is shit. HN constantly complains about Slack/Hipchat/etc "Reinventing the wheel". "We have IRC and XMPP" they cry without stopping for a second to understand that IRC/XMPP as not trivial to setup correctly, mobile support is complete shit, without things like bouncers or servers to hold all your messages you can lose things and stuff not be in sync, clients can't really add features unless all client can support it, and more.

Does it kind of suck that the dream of federated chat died/is dying? Yes but if it doesn't support what users really want then it's the standards fault. I don't blame any of the major players for pulling support, it wasn't going to move fast enough so they ditched it and built something that would.

XMPP (like many other similar protocols) is failing not because it's "not moving fast enough", but because open standards are required to make a commitment to simplicity (in order to be inclusive towards as many devices as possibility) which, in many cases, directly opposes the requirements of a business (ie. having unique features.)

In the end, it's quite simple: federation and simplicity work _against_ many business cases. This is why this stuff is being dropped by the big players in favor of proprietary solutions left, right and center.

Push notifications, message syncing, read receipts, offline sending (as in I'm offline, you send a message, I get it when I log in), encrypted chat, mobile clients, desktop clients, chat room support, direct message support.

These are the things I need out of chat and while XMPP had extensions for some of those things finding server software and client that supported it reliably was near impossible.

Google really betrayed XMPP effort, and Eric Schmidt should be blamed for it. Others aren't any better - just greedy gardeners of their walled gardens.

Reply on news.ycombinator.com