Jabber.org has migrated to Prosody IM | Hacker News Reader

Jabber.org has migrated to Prosody IM

jabber.org

160 pointsneustradamus78 comments

If this news has anyone tempted to dust off an old jabber.org account, you may get an "account disabled" notification when trying to log in. Try again in an hour or so and you should be able to log in successfully.

Why? We've had well over a million accounts registered at jabber.org. We came up with a hacky solution to speed up the migration: we initially imported all accounts that have been active in the past 6 months. Now we're importing them on an on-demand basis. If you attempt a login and your data has not been migrated yet, you'll get "account disabled" error but your account will be prioritized for import during the next migration run (which is now running asynchronously multiple times per day).

Also note that the migration this weekend was just the first step in bringing the service back to life, it's still lacking a number of modern features that are widespread elsewhere in XMPP these days (for example, no push notifications for mobile clients). We'll be turning these things on over the coming weeks.

Finally, once the dust settles, we'll be looking towards the future of the service. Potentially opening up registration again (closed since 2013) and options to help the service be more self-sustaining (such as accepting donations).

Does the disabled-message also tell you to come back in an hour? If not, you risk people trying, seeing their account is deactivated and never bother to come back.

It does not. But if they haven't used jabber.org in over 6 months and then randomly try to log in just once, they're probably not that desperate. Either way, their account will be imported and waiting for them when they try in another 6 months :)

I can’t speak to how tough it might be to edit and re-localize the generic error message here in your system, but I’ve seen many instances where now-loyal (B2B) customers saw error messages, and the wording of the error, especially in providing a path to escalate, was critical to turning that into a positive experience. Bad error messages, in a way, should be treated as a serious (though not critical) bug IMO.

They will think their account got deleted or hacked or they forgot the password. Probably never coming back ever.

or they go to www.jabber.org read the message and log in later

You expect users to read? (Not sarcasm, btw)

Add that to falsehoods about users that developers believe.

So they are using the jabber.org client to connect, and know that's who controls the underlying protocol?

Or did they just install a random client?

You can install a random XMPP client: - https://xmpp.org/software/clients/

What was it running before being migrated to Prosody IM?

It was running M-Link from Isode: https://www.isode.com/ . Their focus and expertise is mostly military and government deployments, rather than public IM services. We are extremely grateful that they offered to sponsor the service with both M-Link and hardware back when we really needed it, and it has been rock solid for many years.

But we're looking forward to the flexibility we'll have with Prosody to make some changes to the service and add some features we've been missing that are understandably not on Isode's roadmap. For example, we've had registration closed for many years, but we're considering experimenting with invitation-based registration: https://blog.prosody.im/great-invitations/

How's Kev these days?

The XMPP server software history used: jabberd -> ejabberd -> Isode M-Link -> Prosody IM

did you at least put a responsible error message for all people with failed logins to mitigate the issue?

Why not just seemlessly migrate right then?

That would require code to do that.

I'd guess that by the time they got that implemented and tested, they'd no longer need it.

Very happy for Matt and the Prosody team. This is further validation of their good work.

We use Prosody successfully for powering the signalling in Jitsi Meet. Its extensibility has allowed us to develop many many features easily.

My one experience with Prosody was writing a plugin to extend Jitsi Meet to show a list of the available conferences and their participants before you join (never got around to developing it beyond the prototype stage, but at least a deliverable that came out of it is a thorough explanation of how to use lib-jitsi-meet that I felt was missing: https://github.com/solarkraft/lib-jitsi-meet-demo - I still believe the feature to be important for allowing spontaneous meetups -don't you want to steal my idea?).

What the hell, I hated it. The parts I needed were super under-documented and Lua/their implementation of it lacks so many features normal languages have - I had to copy a lot of things together from the web to answer basic questions like "what properties does this object have".

To their credit the people in the chat room were very responsive and helpful; I probably should've asked there earlier instead of making sure I don't ask a redundant question.

How do you develop your Prosody plugins? Did I miss some obvious way to set things up so that it's actually convenient? I used the Docker Jitsi Meet setup.

And, while you're here, maybe you could explain the reasoning behind break-out rooms not just being their own conferences (or so it seems to me).

Prosody developer here. Sorry that your first experience developing with Prosody was a poor one. I think about this a lot - we've had a lot of people come from the direction of Jitsi, wanting to extend things for various purposes, and I understand that, depending on what you are trying to do, it can be like jumping in at the deep end.

All at once, not only do you have to suddenly grok Lua and Prosody's APIs, but also XMPP and the way that Jitsi uses it. Prosody's API documentation typically assumes you understand XMPP concepts and what you are trying to do with them.

I also generally only hear from people after they've already tried and struggled (they either reach out then, or I hear the story later on after they've given up). Unfortunately this means we don't get much useful feedback from folk like you who are trying to build upon the Jitsi/Prosody duo.

I've thought about some Jitsi-focused developer docs, and possibly even an additional API layer that presents things using high-level Jitsi concepts rather than the low-level XMPP concepts. But I don't know how much demand/interest there actually is, or whether I'm even barking up the right tree with these solutions.

Anyway, this is something I'm aware of and I appreciate any specific feedback - you can email developers@prosody.im to reach us.

Thanks for responding!

I did find it a bit hard to understand how Jitsi Meet works with XMPP and documentation on that would make development much easier (finding the right events to listen to, for example). Understanding XMPP was a bit of a hurdle as well, but I found it rather easy to clear (just some effort) because the specification is very clear.

The hardest parts were really the Prosody bits. "Why does my plugin not load", "What the hell does this method return", "Which methods are even available", "So how the hell do I actually use this WebSockets plugin now" - stuff like that.

I don't remember the specifics very well because I did it a while ago, but few end-to-end tutorials with tips on finding the answers to my questions (where in the code do I have to look/what can I infer from it?) would likely have helped a lot.

This interaction may just encourage me to have a look again. I'll keep in touch this time, thank you for caring.

I don't think the difficulty of the Lua language is the issue. Lua is very easy for web developers (same abstraction level as JS). The issue is that a great number of Lua developers enjoy abusing the metaprogramming functionalities for basic APIs. Not everything needs to be a meta table, a simple method call or getter/setter function is more than sufficient.

I really like Jitsi Meet. In about 30 minutes I went from nothing to a fully working Jitsi Meet server and a fully working Excalidraw whiteboard! Now I use Jitsi Meet all the time for quick discussions with random people about all sorts of topics of interest to me.

https://www.youtube.com/watch?v=4rlffwHUchk

Granted it took me months to get to this point being new to Linux and not knowing anything about Prosody, Jibri, or even what a repository is. :D

I agree. Matt and his team are very active in the community helping others implement Prosody and make use of XMPP. He even took the time to provide me with helpful feedback a few years ago[1] when I took my first steps with XMPP.

[1] https://samirparikh.com/blog/this-is-how-you-run-an-open-sou...

Thank you for Jitsi. I use it all the time.

There's actually a nice service that lets you make and receive phone calls and texts over XMPP: https://jmp.chat

That's a really cool service! Pity they don't offer European phone numbers...

JMP.chat is a commercial provider, but their infrastructure is open-source. The central bridging component is actually cheogram.com (maintained by the same team). If you're determined enough, there are other providers you can bridge through Cheogram, including Vonage and Twilio, which offer European numbers. It's not as seamless as JMP, but if you're determined enough it works: https://wiki.soprani.ca/VonageSetup and https://wiki.soprani.ca/TwilioSetup

Alternatively, hang tight... I know JMP do want to offer European numbers at some point.

Does anyone know if beta prices will be honored after beta completes?

What happened with XMPP? Seems like the fragmentation of extensions kind of broke the client/server ecosystem, but the fundamentals seem solid, used by Whatsapp successfully, but for some reason recent open messaging alternatives seem to decide to develop new protocols rather than build on top of XMPP.

The client/server ecosystem in XMPP is going strong. Fragmentation is no more of an issue in XMPP than any other open ecosystem. That's not to say it's easy to prevent things being fragmented in an open ecosystem, but it's something we try to stay on top of.

For example, for some time now the XMPP Standards Foundation annually publishes the "compliance suites" which detail for developers what up-to-date implementations are expected to be supporting. We recently updated the xmpp.org website to automatically calculate and display the compliance status of every project.

This, for example, is not fragmentation: https://matthewwild.co.uk/uploads/screenshot-20230119-167413...

Monal's support for calls is in alpha builds, and they're hoping to release it in the coming months. Gajim (written in Python) is looking for someone to help them out with their audio/video call stack, which needs some attention.

Servers (as in, actual deployments) are monitored via https://compliance.conversations.im/ - which feeds into curated lists such as https://providers.xmpp.net/

> Seems like the fragmentation of extensions kind of broke the client/server ecosystem

It did. It was bigger and bigger mess, which feature in which client was supported by what server. I still remember how spotty basic stuff (nowadays) like "send file to someone" was.

And then corporations happened. There was a brief beautiful period of time where you could just put up your XMPP server and chat both with Google and Facebook users directly, but companies figured out closing down their gardens is more profitable and it was no longer the case

> but for some reason recent open messaging alternatives seem to decide to develop new protocols rather than build on top of XMPP.

Coz it is a fucking mess of XEPs spottily implemented between clients and servers, and not always that user friendly.

If you can control server, protocol and client you can just implement stuff. You don't need to make a spec, hope other people's client implement it, start using it in your server, and hope everyone else agrees on it too. And you can just deprecate stuff that turned out to not work well, or just need to be done differently.

Frankly, XEPs shouldn't exist. There should be just XMPP 1.0, 2.0, 3.0 etc. with maybe optional "XMPP voice comms", "XMPP video comms" pack of features (for those cases where full voice/video chat is not relevant).

You either implement all or are noncompliant, no more "well, you send a message but target client doesn't support this XEP, they get garbage"

Breaking the existing deployed network every time a new feature is added is unfortunately not a solution for an open ecosystem. While there are obviously limits to how far backwards compatibility should go, it doesn't make sense to deny users basic messaging just because their app doesn't implement calls for example. That would just cause a different set of frustrations.

I wrote a bit about this dichotomy in this blog post a couple of years ago: https://snikket.org/blog/products-vs-protocols/

> Breaking the existing deployed network every time a new feature is added is unfortunately not a solution for an open ecosystem.

I didn't say it but yes, the support for old version for some time would need to be a thing. But it should be defined (say "3 years for last minor release of every version") so it can be planned for

> While there are obviously limits to how far backwards compatibility should go, it doesn't make sense to deny users basic messaging just because their app doesn't implement calls for example. That would just cause a different set of frustrations.

As you clearly didn't bother to read before you answered, here is what I mentioned in comment you didn't bother to read before answering

> with maybe optional "XMPP voice comms", "XMPP video comms" pack of features (for those cases where full voice/video chat is not relevant).

I don't argue for single version, but to very small subset of featuresets instead of sea of XEPs

XMPP, to my knowledge, pioneered the concept of a micro-kernel standard with feature extensions and profiles to indicate standard sets of extension support. It was, and is, a great idea for many standards. Unfortunately, a number of servers and clients ignored the profiles and so things went the way they did. XEPs were a great idea, in my opinion.

I don't think so. Having a bunch of experimental extensions that then can be tested in the wild and rolled to next version of "the standard" would be better flow IMO. Current flow just added a ton of fragmentation to the ecosystem

So you don't need feature matrix, you see "client supports XMPP 4.0", you know what will work and what will not.

I do like Rust development process - stuff gets added to experimental, lives and gets tried by actual users, gets feedback, gets changed and then lands in stable. So stuff looking good on paper but... not being good ends up quickly weeded out.

I wish it would come back , I really despise having 20 chat apps.

I remember the good old days, when I had one client to talk to my friends, no matter which service they were on. The services themselves were a commodity, which I guess is exactly why this is no longer the case.

This is why I use my Matrix bridges. XMPP has bridges too, of course (that's what the Matrix bridges were inspired by) but it seems the "up and coming" status of Matrix helps the bridges be maintained better.

FWIW, the EU has passed the Digital Markers Act which requires big tech companies to open up their platform. It'll take a while before it takes effect, but unless Apple, Google, Meta and all others are going the very expensive "fine me and we'll see what happens" route, we should soon see movement towards an environment where different apps can talk to each other again.

The tech industry being what it is, there is no real standard for interoperability yet, though there is an IETF taskforce working on setting up a standard companies may follow: https://datatracker.ietf.org/wg/mimi/about/

This, critically, considers encrypted messages in a standard way so that we don't lose E2EE in the process.

Let's hope this ends up being good, I can see it taking twenty years and at some point standardizing a backend into the encryption :(

It also wasn't difficult to set up your own server and bring your own domain, resulting in an easy to remember vanity handle like chat@lastname.tld. I hosted both prosody and ejabberd for years for friends and family, and it was orders of magnitude easier than self-hosting a mail server. Sadly, eventually everyone moved to proprietary platforms because file sending worked badly in those days, and mobile XMPP clients were simply terrible (no push notifications and such).

Alternative are cropping up. I've been testing out solutions with other nerds.

https://www.beeper.com/ is promising. It's set of matrix bridge over 15 chat network or so.

Neat.

I've been successfully using their iMessage bridge for my personal server for about a year now. It's very nice to use iMessage on my Linux machines, since that's the chat network everyone around me has decided to use.

Is this in accordance with those services’ TOS? How can you be sure you won’t get banned?

For instance, I remember hearing that Discord banned users who used alternative clients.

Good point. From my pov they can then shove their service up their ass but it’s a valid concern.

I really despise having 18 proprietary chat apps.

20 feels like an exaggeration, but there are more than 20 Matrix bridges, so maybe I'm just less spread out than some. I've been using Telegram, Signal, Google Chat, and Discord so far, on a self-hosted server. It's not too hard to stand up, especially if you hang around here, and works a treat. Highly recommend if you want one app to message everyone from.

+1 I just set up a Matrix homeserver for the first time, and it was really slick with the Ansible playbook.

I am astonished by how well the Signal bridge works — everything works! Read indicators, threads, mentions, groups…so far I have it configured as a linked device to my phone, but you can also configure it as your primary device.

I really want to use the WhatsApp bridge, but that seems messier. I don’t want to install WhatsApp on my phone, so it requires running Android in a VM with a VoIP number, and apparnetly risks getting banned from WhatsApp.

It was a terrible pile of overengineered sophomoric semantic-nonsense second-system effect garbage from day 1. The need for everything to be an extension and XML was a millstone around every client and server's neck, the XML validation is directly blamed for the downfall of Google Wave but it also seems like that drag was the primary reason why the client experience always sucked. The reason the only successful uses of XMPP are in closed ecosystems isn't nefarious companies trying to control it (although there are those), it's because the only way you can actually get a reliably compatible experience for the kind of rich chat that everyone expects these days is if you control the clients and servers together.

I genuinely think the main reason there are no successful open-source chat apps is that XMPP dragged them all down.

"recent open messaging alternatives seem to decide to develop new protocols rather than build on top of XMPP"

XMPP is reasonably specifically a protocol for Instant Messaging. You can theoretically do whatever you want with it, and you can theoretically turn it into a mechanism for delivering social-networking style asynchronous messaging, but it doesn't necessarily bring a lot of value, and it makes a certain amount of fundamental architecture decisions that aren't necessarily the best for a social network.

I am interpreting your "recent open messaging alternatives" in the social network framework because my perception is the major ones you might be thinking about are those things, like Mastodon and such. I'm not aware of hot new open IM protocols that are just "old school" IM.

What makes something like Signal, Telegram, WhatsApp, et al not fall into your definition?

That seems to be what the parent commenter has in mind far more than Mastodon, at least to me.

Not what I would personally consider "recent". Your mileage will vary; there is no objective definition of that.

I would argue since the comparison is XMPP, which has been around for almost a quarter of a century, “recent” has a slightly wider definition than most times.

They're probably referring more to Matrix, which does have a somewhat different scope.

I used xmpp for a while and it has a fairly good hosting experience (easy to setup and maintain) but it lacks good clients. At least clients that the typical whatsapp user would like. And now, that everyone pushes matrix as the best alternative to everything, the situation will likely not change.

My family were WhatsApp users, and I moved the family-family chats from WhatsApp to Snikket (XMPP) a few years ago. They still have WhatsApp installed, but consider Snikket the go-to for reaching family members. It's not a WhatsApp clone, but they haven't struggled with it.

Nothing. It experienced some stagnation around the time Google ditched it for Hangouts, but it's been going strong again for years now.

I've been using it comfortably for years now, what do you mean?

What do you mean "what happened with XMPP"? Jabber uses XMPP.

But approximately nobody uses Jabber. He means, "why did XMPP fail to achieve the level of adoption it seemed poised to take a decade or two ago?"

If you use Cisco phones, you use Jabber.

Cisco makes phones?

Yes, very widely used enterprise VoIP.

Edit: At least in North America, can’t speak for the rest of the world.

Have you literally never been in an office? They are probably the biggest manufacturer and supplier of telephony systems, at least in the UK. Well, since VoIP came along, anyway, before that it was all Nortel and BT Meridian, with the odd Avaya here and there.

The VoIP server software they use supports XMPP natively so your softphone on your PC is just a Jabber client, and (as far as I've been told) they use XMPP to sync between servers.

You've definitely used Cisco telephony kit, because you've definitely heard this tune: https://www.youtube.com/watch?v=pais41IW5dk

yes in fact they made iphones before apple. (apple stole the name and later licensed the trademark from cisco)

> What happened with XMPP?

There never was meaningful money behind it. It seems the only way a protocol can ever grow today is by having for-profit companies sponsoring its development, doing the marketing, pushing it to individuals, companies and public services. That's not too hard to understand: at some point you need people working on it, and in a capitalistic society you need to pay them, and thus you need to somehow make money. Even with all the goodwill and proper licenses and all, you need massive humanpower.

Google, Whatsapp, Fcaebook all used a form of XMPP but never intended to propel it as a federated protocol, only for their own usage.

Well, I can't say we didn't try.

I remember how excited my boss, Andre, was when he discovered the Jabber protocol. He convinced Webb Interactive to spin off Jabber.com as a subsidiary, and I went with him. Our first space was the former office of Webb's chairman, with four of us in there. Later, when the company moved from Glenarm Street to Wynkoop Street, we had part of the floor above Webb's floor. I spent a lot of time working on the bridge from Jabber to ICQ, which was always kind of touchy. And we did have a nice booth at the O'Reilly Open Source Conference in Monterey.

Then came the dot-com crash, and Andre was forced to lay off a bunch of people, including me. He didn't want to do it; I think he was as much in tears as I was. Eventually, I understand, Jabber.com was bought by Cisco. And Webb Interactive eventually kind of sputtered out.

I had some rough times, but I'm doing a lot better now after a bunch of other job moves (not to mention a gender transition). Andre's got Ping Identity now, and he's finally having the kind of success he deserves.

Oh I'm sure some companies did try, and probably even deserved success, but like all businesses there's always a question of chance involved. Maybe your company was too early to the game, trying to push computer communications when not enough people used computers in the first place; I don't know. But I would have loved to have had a proper communication protocol 10 years ago instead of doing the same thing over and over and over again, even today.

Prosody (server) and Monal (iOS client) make a great combination. I'm not aware of anything for Android that's as elegant as Monal but I'd be happy to hear people's suggestions.

Conversations [1] is the best Android XMPP client I know. IIRC they pushed the adoption of OMEMO and implemented it first, before the desktop clients could catch up.

[1] https://conversations.im/

Conversations (OSS) is usually the gold standard on android.

On iOS at least, I find Siskin to be more elegant.

Honest question, why not ejabberd? What was the +/- for prosody?

Valid question! Honest answer: the folk involved (including me) have more experience with Prosody these days.

Jabber.org previously ran ejabberd for years, in fact that's what it was running when I joined the admin team (I was also running ejabberd on my personal server at the time). We had quite a few problems with it back then, and for various reasons decided to switch to something else to help bring some stability to the service. This is all in the distant past (literally 10+ years ago), and I know for sure that the several problems we kept encountering on jabber.org have been fixed long ago. Many other large XMPP services run ejabberd successfully, including conversations.im.

But now that Prosody is more mature, the team has more experience with it, and it has a few more features than ejabberd that we'd like to support, it's what makes the most sense for us right now.

If you're trying to decide between ejabberd and Prosody, they average out to being equivalent in terms of protocol support. ejabberd has clustering, and a commercial option for people who want that. Prosody has a strong focus on extensibility, and has hundreds of community modules at https://modules.prosody.im which provide various kinds of extra functionality.

I don't think either project is overall "better" than the other, but each has strengths and weaknesses for specific use cases.

Thank you very much for the answer!

For me the solid clustering and erlang was the main-point to choose ejabberd.

I run a prosody server for a small group of people. It's not hard to set up, though the XEPs are often poorly documented.

The only significant irritant for me is spam. Open registration servers are the blight of XMPP.

In case you haven't seen, there are a number of things you can do about spam: https://blog.prosody.im/simple-anti-spam-tips/

Great to see open standards services switching to open source software.

Reply on news.ycombinator.com