Instant messaging: Protocols are "commons", let's take them seriously
process-one.net
process-one.net
And that’s a problem!
After about 30 years we see a clear pattern: something like AIM comes out, because it is locked in there is no competition for the client or the server so there is no motivation to improve either. The thing is the computing world moves ahead so the system goes bad like cheese as the environment it works in changes. A decade later there is a new client out and people are saying “Remember when AIM used to work? There’s a new service that works just as well as AIM used to work” and finally people are so burned out of AIM they are ready to switch but failing messaging systems can lumber on for years like that.
Emphasis on “just as well” because in this scenario the systems are not getting better, you’re not getting an upgrade by switching, you’re just treading water. What people aren’t realizing is that if we had interoperability we would have had competition and after 30 years we’d see 30 years of progress in messaging systems unlike the occasional churn but lack of progress we’ve had.
Modern messaging systems allow me to do much more than AIM. I can send and receive very high quality photos and other files. I can video and audio call all around the world at zero cost. I get very little spam on WhatsApp/imessage.
> What people aren’t realizing is that if we had interoperability we would have had competition and after 30 years we’d see 30 years of progress in messaging systems unlike the lack of progress we’ve had.
We had interoperability with phone numbers, but I saw no improvement until AIM/Skype/MSN Messenger/Google Talk/WhatsApp/iMessage came along.
With the previous “interoperable” system, I had to pay for international audio calls and SMS/MMS, so I never did.
So interoperability does not seem like a guarantee of progress, and things might even stagnate due to interoperability requirements. Pros and cons for both.
That's a really huge "if" right there.
Last week I was playing through Asgard’s Wrath 2 on the Meta Quest 3 which is a AAA game for standalone VR which has the scope of a big console release. A lot of people use VRChat already. I could see the Apple Vision Pro being ablem to synthesize a very good avatar with its eye camera.
High-end video games is one of the niche uses where I do expect fairly widespread adoption.
They were always destined for being retired once smartphones became a thing.
Indeed, there wasn't synchronization between devices, although there are many ways that could have been handled.
I used Trillian for almost a decade; it was the second best solution to actually being able to message Yahoo and MSN users from my AIM handle.
I think people have been bothered, but when an entire platform disappears, there's not much anyone can do. The coding wizards who are capable of doing something (in concert with some marketing wizards, maybe) don't seem interested.
So it's really about what those who have money they spend paying for development are interested in, I think, not the interests of "coding wizards", who might love making something different if they could be paid similarly (or at all?) to do it.
I don't think it's controversial to say that there's generally a focus/priority issue in the tech industry. The world's most profitable progress-themed ad distributor.
The number of developer-hours getting paid to work on proprietary messaging systems is pretty huge.
And this is hardly irrelevant to understanding why we have proprietary messaging systems.
Whether you personally are a developer or not -- if you have spent more hours building something you haven't gotten paid for than you have at your paid job, you are an unusual person. (Such unusual people do exist! they often, but not always, have atypical lives in some way that allows this)
I mean, you think "the world's most profitable progress-themed ad distributor" comes about becuase "coding wizards" are just intrinsically intersted in this, rather than because it's what makes people lots of money? It's just a lucky coincidence lots of people made money, that focus is really because "coding wizards" by and large just love distributing ads, it's a personal interest? This is all guided by the personal interests of "coding wizards" rather than by who pays for software development for what?
Not at all. The "most profitable" part was an acknowledgement that money is at the center of such decision-making. But that paradigm exists alongside the one where FOSS and indie libraries and software are at the base of a number of important technologies. I'm just having trouble understanding why messaging isn't one of them.
Counterpoint: IRC
One google engineer on his personal time introduced entirely new systems into IRC (RobustIRC) https://robustirc.net/ which is the most commercial support that exists.
The IRCv3 initiative is a very small number of people, none of them supported commercially.
I agree what it gets the consumer is crap.
XMPP was that, and it saw widespread adoption up until Facebook and Google walled off their XMPP federation- slack even disabled its XMPP bridge (that wasnt federated).
Companies definitely killed that open protocol, though there could be arguments against XMPP: functionally they could have been worked around, there is just a significant financial incentive to not want to.
still alive and well, survived takeover attempts, see freenode -> libera.chat and got a shiny new protocol spec with modern features
Last time I checked user numbers were both steadily in decline, and small in absolute terms (top 10 IRC networks combined have ~500k users - compare to eg discord with 150 million users. There are individual discord servers with larger user numbers than the entire world of IRC)
> survived takeover attempts
Didn’t the takeover succeed, and then all the users migrated to a different server? I guess if you’re saying “IRC as an abstract concept survived the takeover of an individual network” then sure that happened.
> got a shiny new protocol spec with modern features
Perhaps I’m reading this chart wrong, but checking the IRCv3 feature support page, features that I would consider “modern by 80’s standards, table stakes by 90’s standards” (eg account registration, chat history) are still in the “draft” stage and not supported by libera.chat. Features from the 2000’s (eg mobile notifications) aren’t even being drafted?
If interoperable protocols really drove progress, Discord wouldn’t have attracted more users in 6 months than IRC did in 30 years :P
All the cool messenger tech that I want to use is buried under user experiences that drive off normies accustomed to iMessage. I have made peace with it and use Signal instead.
> ...if we had interoperability we would have had competition and after 30 years we’d see 30 years of progress...
SMS has been popular and interoperable for at least two decades. The progress we've seen is MMS and the promise of RCS.
The problem with building interoperability is that the innovators are innovating for their competitors.
It kills their ability to add differentiating features, it removes their networking effect moat, and it provides them with no benefit. It's so much more appealing for their answer to be "just download my app".
It happened for telephone and email because they were developed in a different era. It's not going to happen for IM because we're in the era of "why build it if I can't profit from it?".
Who is going to pay for the bandwidth for 5MB to 100MB photos/videos/gif/files that are sent through the network?
> "why build it if I can't profit from it?".
Currently, it seems like a variety of entities are subsidizing this via things like peering agreements and whatnot to make the use of internet simple for the end user.
The cross-protocol clients back in the day used reverse engineering to work. That would not fly today because today's tech culture is less hacker friendly, more litigious. Such a company would not receive venture funding due to its risky nature.
I think there is room for improvement in messaging, but interoperability makes it more difficult. I think it is in everyone's best interest to first ask for better features, let its inventors make a profit, then worry about interoperability once competitors catch up. If the market is competitive, interoperability can be achieved through coalitions. IBM's MCA vs Compaq et al's EISA is an instructive example.
Which is why you don’t see successful messaging companies rushing to open their systems.
What company wants to spend the effort to build a successful messaging platform, only to open it and see some nobody come in with feature parity _and_ the ability to reach/convert every single one of their customers?
Open protocols would have worked great if that’s what was mandated _before_ these established products were in the market. Now, it’s going to be an uphill fight all the way, and companies are going to do the absolute bare minimum—-not just what they can get away with technically, but what they can get away with legally (e.g., Apple adopting RCS so they can point to RCS as the common messaging denominator, thus avoiding opening up iMessage itself).
It'd be better for people (maybe not apple), and the people can decide how their country is run, including requirements on tech interoperability for businesses operating in that country.
There are plenty of countries out there that preference corporations over people (I think that's the technical definition of a fascist government). One is free to move there, if they find democratic rule so distasteful.
Personally, I prefer majority rule rule to minority rule, given a choice between the two, and likewise prefer rule by the people to rule by a minority of corporate "persons".
What iMessage can’t do is translate a lot of the QOL and UI features, each client will have to adapt to read those on their own.
I used to see the same thing with phone numbers: a stack of phone (desk phone), fax, pager, voice mail, mobile, all eventually collapsing into a single phone number.
Why did this not happen with messaging (which is more than 30 years old, despite what the article says)?
Those .ARPA names were part of a lengthy transition to the DNS; before this I was just gumby@mit-ai or just gumby@ai
The ARPA. TLD isn’t used for hosts any more but has been repurposed mainly for network database lookups. Yes, the `.` after the TLD is there for a reason.
So many hours tweaking Sendmail m4 logics for ARPA to DNS and back.
WhatsApp is another silo, even though it uses phone numbers as identifiers.
I sent someone a business email the other day. They told me I got lucky they even saw it, as they now communicate entirely via Slack. Slack isn't even trying to be an email competitor and it still wins.
SMTP is still useful for cases where the additional feature-set of other platforms isn't important but a globally unique ID is.
Not making protocols interoperable allows the app to attempt to build their own walled garden.
Interoperability is a very old problem, and it has been reasonably solved a few times (jabber pre Cisco/xmpp/etc) and most recently technologies like matrix/riot are pretty well positioned.
While tools like pidgin exist today, software like trillian of old allowed one interface to log into everything.
I am getting increasingly concerned about the number of times aggressively antagonistic things are defended because addressing them would potentially "stifle innovation".
Innovation is important, but it's far from the only important thing. We need to stop sacrificing everything on the altar of "innovation".
"Woe is me, this will stifle innovation" cries the megacorp, as their latest merger is blocked. The amount of people that are sympathetic to those cries are alarming.
XMPP in 2023 is having all the modern features that you can expect while being massively scalable and having a really nice ecosystem (there is still some quirks here and there but it really improved the past few years).
This was ages ago, is pidgin still one of the best xmpp clients, or are there better ones now?
I remember that pidgin even used with facebook, unfortunately facebook dropped xmpp as someone else in this thread mentioned!
[0] https://media.ccc.de/v/37c3-12064-rfc_9420_or_how_to_scale_e...
This never works. It's a recipe for disaster. The security model of a secure messenger informs the protocol from top to bottom. You try to retrofit cryptography onto a general-purpose messaging protocol and you get group membership protocols that allow servers to decrypt messages, and centralized databases of everybody who's talking to everybody else. Even good, purpose-built protocols have ended up with these kinds of problems (see Matrix). Never do this.
However, it could be better: first we have been converging on a single Rust crypto impl (at last) - see the crypto section of https://matrix.org/blog/2023/12/25/the-matrix-holiday-update.... Then we’re shifting to TOFU (so unsigned devices never get encrypted for). Then, finally, we’re shifting to client-constrained group membership (unless MIMI gets there first, and then we may switch horses to their MLS-based solution).
edit: the other issues raised by the nebuchadnezzar paper were impl bugs in the 1st gen crypto impls, which were fixed at the time. also, the premise that you can’t add encryption posthoc to a protocol is flawed, imo. Matrix may not have nailed it, but as long as you consider the crypto as if it’s a new protocol and have the agility to evolve to it, everyone wins.
edit 2: there is no centralised db of who talks to who in Matrix. individual servers can see who shares a conversation which their server participates in, but this is very hard to avoid given you could always figure it out via traffic analysis (unless you go down a Nym style mixnet solution).
For example, you can refuse to use Google Chrome, and instead choose Firefox or Vivaldi. Your web experience will be slightly different, but the most important parts will remain the same: You type an address, you wait for it to load, and you access the content.
On the other hand, refusing things like WhatsApp means there's a non-insignificant amount of people that use WhatsApp to communicate exclusively. This may not have impacts for you, personally (although I would be hesitant to believe that), but it definitely leaves out billions of people who communicate exclusively via WhatsApp.
A similar thing happens, for example, if you refuse to use YouTube, which is the largest Internet video platform on the planet: You will have to refuse to watch any content that is only uploaded to YouTube, or put up with frontends that use YouTube in the background, or perhaps even be forced to pirate videos, neither of those three options is good for different reasons.
That's not a problem for me at all, honestly. I interact with my friends and family through other channels. Nobody I know uses these services exclusively.
> it definitely leaves out billions of people who communicate exclusively via WhatsApp.
I don't want or need to communicate with billions of people. I want and need to communicate with the people I know and care about.
Don't get me wrong -- I'd use an IM app for the convenience if there was one that served my needs. But as long as each IM app is it's own island, then none of them serve my needs at all. I'd have to have a half dozen of them, and that's unmanageable and ridiculous.
Google talk used to use xmpp as well back in the good old days, and there's maybe even some more!
It'd be nice if open source projects where to culturally make it a norm to use xmpp instead of slack, discord, and the like, sort of how everything used to be run over IRC in the even better old days!
We do have common protocols: email, IRC, XMPP, Matrix, Signal, MLS, all of which are various good degrees of well-specified and open source. We have the tools, we just essentially need to be OK with saying "WhatsApp is just for interacting with my kid's teacher; my friends and family use Signal". I think nerds, geeks, punks, and artists forget we're the interesting ones. It's fine to use Threads for sports/normie stuff and Mastodon/Signal for our stuff. The separation is a gift.
SMS is about as close as exists for open instant messaging, but it requires a phone number
IRC seems to have inspired Discord and Slack, with the closed variants improving on features and discoverability.
As far as I know, XMPP doesn't really have a cross-server ability like SMS and IRC, though.
Edit: there was also a time when AIM could send and receive SMS due to this structure, and at one point it was trivial to set up a script to email your phone number as SMS for alerts and such. This became more complicated as SMS providers started to require a lot of the same stuff as email providers to cut down on spam.
E-mail is totally intercompatible, but the experience for anything apart from "the equivalent of letters" is simply horrendous. Delta Chat tries to make e-mail more like a chat app, but it isn't perfect, because e-mail wasn't designed to be a chat application.
XMPP has other massive usability flaws. So does IRC, Matrix, and others.
Meanwhile the clients Element built to bootstrap the protocol have been stuck paying off huge amounts of product debt, and are finally at the point now of surpassing mainstream apps - eg https://element.io/blog/element-x-ignition/ - but it took years longer than it should have to get there. Again, it’s not the protocol’s fault though, other than the extent to which building a protocol draws energy from building killer apps.
2. Even though there are so many services, I find functionality to be lacking on many of them. I don't want to use facebook, because of privacy. I don't like signal because it doesn't give you your chat history across devices. I used to love hangouts, but google destroyed it. Many of the services are tied to your phone and don't work if it isn't on. There is a good chart in german comparing the different features. I just use imessage because it's end to end encrypted, i trust apple's privacy and it works.
The law has already passed and will apply to messaging services with more than 10% of the market. WhatsApp and Facebook Messenger will need to comply. After review it was determined that iMessage doesn't have enough users in Europe at this time.
[1] https://www.theverge.com/2023/9/6/23861030/imessage-bing-eur...