Use protocols, not services
notnotp.com
notnotp.com
It turns out it's very slow to evolve a protocol. How long did it take for IRCv3 to handle channels having persistent history? How about channel takeovers via network splits? We knew these were problems in the 20th century but it took a very long time to fix.
Oh, and the chathistory Extension is still a draft! So is channel-rename! And account-registration?
And why is it still so painful to use Mastodon?
That's but one of many examples. Consider how the consolidation of HTML and HTTP clients was the only way that we ended up with any innovation in those services. People have to keep up with Chrome who just does their own thing.
I want to want a decentralized world governed by protocols, but good software that iterates quickly remains the exception rather than the rule.
That is good to understand, but when that trade starts causing issues, it is important to remember that there was a trade made.
We aren't as stuck as we think we are, unless we decide not to reevaluate our past choices.
Matrix has shown how incredibly difficult it is to make a modern service in a decentralised way. Requirements like preventing spam become immensely difficult.
Do any fully trustable ID validation services exist? Ones that verifiably never store your ID but just a validity status for a given ID on a blockchain?
Imagine a messaging app for example, a 1 month old account with a Nigerian phone number cold DMs an account in Australia. The likelihood of this being spam/abuse is extremely high. Vs a 5 year old account that mostly messages mutual contacts cold DMing an account in their own country.
In many countries, phone numbers are a proxy for ID and are difficult to get without having a local ID. The countries which have not secured their phone number system will be less trusted by spam filters.
Assuming you don't want actual ID verification, the choices are even larger but with different trade-offs.
And of course governments attract hackers because they tend to not be up to date on security best practices.
A single abstraction layer on blockchains allows more developers and security experts to contribute and innovate.
E.g. OS exploits were targeted towards Windows, not so much for so many of those Linux distros.
https://hanez.org/document/why-matrix-sucks/
https://forum.hackliberty.org/t/why-we-abandoned-matrix-the-...
https://xn--gckvb8fzb.com/giving-up-on-element-and-matrixorg...
It's pretty good today! Lots of things improved a lot! Some big clean ups!
But think of how much better it would be if people stayed woke, if they didn't just throw up their hands call defeat & say it was never going to work. If there wasn't such a bleak rot in our soul, if we could try to play slightly longer games, I think in the medium & long run it would be much much better for us all.
It feels so easy to spread sedition, to project these fatalisms that only big dumb lumbering central systems win. I'm so tired of this bleakness, this snap to convenience as the only perceived possible win. Let the prophecy self fulfill no more, let us arise from this torpor. A little Ubuntu would be ao good for us all. Ubuntu the old saying (that the distro was inspired by) goes: "If you want to go fast, go alone. If you want to go far, go together"
Oh, TLS also. Encrypted connections over HTTP are trivial.
Arguably this has created far more freedom by making encrypted network traffic default and free. Convenience is also freedom when it comes to accessibility.
TFA mentions both, yes, but as a direct example of service/platform (Discord) vs protocol (IRC, XMPP, etc). The comment asks a question that kinda misses the point of TFA.
Discord could be considered to have "won" in that it's got a lot of (new) users and removes some of the limitations of IRC, but that's _because_ it is a service/platform, and comes with all the trade-offs being discussed in other threads here.
Or one could consider IRC to have "won" because as a protocol it simply can't have some of the restrictions possible with a centralized platform.
It's trade-offs all the way down, but protocols will always have fewer restrictions of the kind currently in the zeitgeist, especially decentralized protocols.
Also the article presents a false dichotomy in my view: protocols need services to be useful to virtually 99.9999% of humans (or at least they do in the architecture we have built since... email?).
Who uses email without relying on servers? Where is your selfhosted email box sitting on if not in a hosting service?
Even IRC relies on servers for people to talk to. I love to experiment with protocols that do not rely on servers - secure scuttlebut? - but even ssb relied on some seed peer that provides a service to initialize the peering
Note that Discord doesn't replace IRC, it also competes with TeamSpeak; there's a whole voice and video sub-feature to it. Not everybody uses it but the fact that it's available in the same software was advantageous to the original market, gamers.
Of course it was also clear that eventually the investors will want to cash out & we are seeing the results of that.
Edit: actually thinking about it - at the bottom of much of it is identity. We need new identity solutions for the protocols.
IRC is pretty good, and it survived the Freenode takeover by simply letting everyone know things are moving over to Libera.
Bluesky is awesome if you just ignore the "Discover" tab, I wish they'd just get rid of it. Librem One did something similar with Mastodon, it was peaceful.
After the initial excitement of finding decentralized platforms like that, I personally realized I don't care much for that type of interaction with people, so I don't use any of them very often. Same way I don't use my phone much, but it's there when I want it. Like a utility should be.
Mind you, we are talking about using these protocols for the general public, not savvy hacker news readers.
also what specifically are you worried about these 100 billion chatgpts doing?
I'm answering to that cost being a problem regarding "what prevents 100 Billion ChatGPTs from using any protocol?" - the context I have in mind for the above being scammers, political manipulators, spam, and people like that using ChatGPT/LLMs to take advantage of various protocols for profit (and the 100 billion figure being a figure of speech meaning "very many").
They mentioned identity being important here. I'm not sure what that means in this context (some kind of cryptographic verification, maybe?), but the part that seems relevant to me has to do with trust. Either a person is trusted by people I trust, or at least an organization I trust makes some claim about this person (e.g. they're actually human, this is actually their real name etc.)
I think we'll be seeing something like that in the mainstream in the not too distant future, for obvious reasons.
I agree that the delivery protocol could be more efficient, but use of JSON is a tradeoff that provides good extensibility and easier parsing (many well seasoned libraries exist in almost every language).
Identities are global and shared across devices. Naturally, if your keys are lost/compromised your identity is lost/compromised.
So the solution they have to this is that your real root identity delegates signing to other identities (generated local to a device) by publishing a note indicating a list of keys allowed to sign on its behalf, and presumably you keep your root identity on a trusted device (like maybe a crypto hardware wallet or a threshold multisig).
But this just reduces the problem and worsens the UX. Your identity still gets lost/compromised if the root is.
There's also an issue with how identity updates themselves work. Since these delegates are really signing for the single root, they need to be synchronized to work properly. There was a common bug (which might still happen) where if you set up your identity on a new device, the app might broadcast an identity update with an incomplete view of your identity and it resets your follows and post history. Since your identity data might be influenced based on every note you've ever sent, and message delivery is unreliable, it's hard to properly sync and reconstruct sent note history. This comes out of a fundamental design issue, where you have multiple "writers" writing to the same state. CRDTs could have helped with this, but it's too late to do that.
This sucks! It forces users to think about key management and has catastrophic failure modes. It's really hard to re-establishment trust after key compromise because there's no notion of identity that lives longer than any one key.
Matrix is not a comparable kind of protocol, but its identity management story is a lot better. Each device has a local key that never leaves the device, and when you add a new device you cross-sign it from another device you have. Homeservers maintain a list of identities tied to a user, and other people can decide to trust the device cross-signing or manually verify each of them. This can be built in a fully decentralized context (which Nostr is not, for what it's worth).
May I ask you to elaborate on this point?
This also means that it gets increasingly more expensive to run a relay as time goes on, making those parties have more sway over the network and giving the ability to selectively remove content.
So that's why I argue it's not fully decentralized, like BitTorrent. BitTorrent does have trackers, but they're only an accelerator over DHT/PEX. Peers can't manipulate content since you independently verify it. There would have to be some kind of in-protocol message exchange directly between participants, bypassing relays, when they were able to reach each other.
I think CRDTs are great, but Nostr has always presented itself as a potentially lossy medium, purposefully. Unlike SSB and Matrix where state synchronization became a complex bottleneck, Nostr is more IRC-like. Relay owners may have to delete individual posts due to legal reasons, or identities may selectively publish different posts to different relays. The devs didn’t see this as a problem since full state synchronization is heavy and requires long term retention of data. I agree that it’s not perfect, the tradeoffs make it harder to reconstruct a full history for a given identity if you’re trying to reach way back in time. But for new content it works really well, and I think this is why they chose this approach. If you publish to a lot of relays, your message will get through to the people who want to see it, although the process is messy.
Yes, but it's a fundamentally unsolvable one due to how the ecosystem has chosen to settle on it. Even blockchain wallets are experimenting with social recovery and hijacking SSO systems because traditional key management is too hard for the average user to do correctly. Users barely want to do key management for that! Much less to look at cat pictures.
> I agree that it’s not perfect, the tradeoffs make it harder to reconstruct a full history for a given identity if you’re trying to reach way back in time.
This is just not how users expect systems like this to operate. If it was purely a low-level async messaging protocol (where retention matters less) that'd be more okay, but it's trying to be used as a general purpose social platform.
And this is why I've concluded that the Nostr ecosystem is just deeply unserious about its philosophy of design and it's fundamental architectural flaws. It's super common to see responses that have the form of "here's why it's actually good that this sucks". I thought it was clever when I first discovered it, but it seems like they're very happy to be stuck with half-broken functionality because it feels fun and janky like IRC and they're all used to the bitcoin ecosystem where they can just blame the user for messing up.
I think that this kind of fragmentation is becoming more common. Not everyone wants to be on a platform with the rest of humanity anymore. And not everyone shares the same design goals for protocols to replace those platforms.
Back in the late 2000s and early 2010s Google and Facebook supported XMPP, so you could login to Facebook Chat / Google Talk via Pidgin through an XMPP gateway (if if this was the default protocol or a bridge I'm not sure, its been years).
The biggest strength I see for XMPP is that because the web and even enterprise (think banking etc) uses XML too, everyone's optimized the ever living crud out of HTML so you could get some very high performance libraries to churn through all those stanzas, but also more importantly, its an extensible protocol. There's no reason it cannot have half of the things that exist on Discord, without disrupting the protocols OOTB design, because unlike IRC and other competing protocols, its extendable by design.
The only example I can think that messages are expressed as documents is Microsoft Teams. And it’s as much an example of what not to do as anything.
Pretty much no messaging apps let you create messages more complex than markdown anyway.
A self-contained rich formatted text file/package that optionally contains attachments or media.Very few messaging apps let you go beyond plain text and let you start embedding media or complex layouts inside a message.
Slack canvases have full layouts including images.
Even whatsapp does formatting, so does signal.
What is a current popular messaging app that does not have rich text features?
Absolutely with you up to here, but...
> Particularly these days when you can have a coding agent write the parser boilerplate, etc. for you.
Absolutely not. Having seen the infinite different ways a naive implementation of XML goes wrong, arguably being one of the main causes of death for XHTML because browsers rightfully rejected bad XML, "Don't roll your own XML implementation" should be right up there with "Don't roll your own crypto".
I don't feel like it's going out on a limb to say that if someone needs to defer to a LLM to implement XML they're not qualified to determine if it's done it right and/or catch what it got enthusiastically wrong.
What I was addressing is, interfacing with an XML parser and converting that into whatever your internal representation is, can be a chore. LLMs are great at that stuff.
My favorite example - Arista network switches can be clients on an XMPP server. Control plane's have to be very slim. XMPP enables someone with a network operator to apply wide, symmetrical configurations across a network, without repetition. You can add the "core" switches to a group chat, and query them for information simultaneously.
Found an example article: https://jonw.mayhem.academy/arista-switch-wrangling-with-xmp...
You would never see Discord as a control plane management option, nor a Slack, Telegram or Signal option. But if all or a group supported XMPP, there would be a low resistance avenue for that (if someone really wanted it).
As it stands, we have product lock in due to each service having it's own system, with limits on interactivity. So I won't be cross-channel quoting outage causes directly from the switch in the company Slack any time soon.
It's an advantage, sure, but to me the serialisation format is the least interesting thing. Others are similarly optimized too. I think the extensibility and approach to standards is far more interesting than the fact it uses angle brackets instead of braces.
Back in the days, I had to write my own parser, existing xml parsers couldn't handle the case well.
I'm very much sympathetic to the post's argument, but I think it should be acknowledged that this kind of claim has an implicit "(for now)" at the end.
The legal system doesn't have good mechanisms for dealing with problems that it hasn't needed to deal with yet, but if most people moved to encrypted & decentralized protocols for communication, it doesn't follow that laws couldn't be amended to give governments powers to legislate or police it at scale if deemed necessary by some sufficiently powerful group (an autocracy, a voting bloc, a national security service, etc)
So I guess the other implicit piece is that one hopes the technological change comes with cultural change to our political expectations - once people get used to privacy and autonomy, they resist efforts to erode those rights again.
Best of luck to everyone advocating for this! Really hoping to see a lot of thriving communities post-Discord in the coming years.
We also need decentralized identity so my identity can exist independently of service providers, but still be owned by me and not an impersonator.
Which for reputable TLDs is permanent, outside illegal activities.
They (like any other entity) can attest, but such attestation should hold as few of any special value as possible.
An unusual position, as historically governments have provided birth and death registries [0], passports, identity cards, etc, etc
[0]: or, earlier, in the West at least, the church
They maintained census, but for government functions (like accounting and taxes), and actual identity communication almost never involved government.
Passports use for anything except international travel is a very modern thing as well.
For most of the history the source of identify was individual themselves (as it should be), that is, one told their name and origin and others accepted that, unless someone knew otherwise.
Or maybe someday we’ll have some interesting revelations about personal identity and sybil resistance won’t be necessary. But that’ll probably be only some centuries later.
I get that identity is a sort of last holdout for the tech libertarians of old. But after years working in KYC, what I saw was the accumulation of vast amounts of sensitive information held by private actors in a way that was completely democratically unaccountable and couldn't be corrected by the average citizen. It's time to bring identity out of the shadows and make it ours to control.
I’m not a libertarian (was; realized why it doesn’t work in reality we have), but I still believe that no entity ever should be able to deny one’s identity, they can only refuse to attest it.
And the more serious problem is that nowadays we’re collectively so much into that flawed paradigm of “identity providers”[1] I’m afraid if a government-ran system happens it’ll would be still built in the same paradigm and engrave that into collective consciousness even further.
Private corporate-ran identities are IMHO better for the foreseeable interim, until we know for sure how to do things right. Because I suspect that whatever we pick as fundamental ideas is going to stick and bless or curse us for a long while. Nation states have longer lifespans than Internet companies popularity, so as weird as that may sound I’d prefer Gmail to, say, that Estonian X.509 scheme (no offense meant; and I’m only considering use outside of government services), despite latter being short-term better.
And - yes - I 100% agree that it’s past the time we should be using proper cryptography for attestation of all sorts, rather than sending passport photos and live selfies to increasingly more and more private companies. But that shouldn’t be general identity verification, it should be only for compliance, only when a law forces to obtain some information from some government-issued credentials. This part desperately needs moderation. But for the love of what’s still sane - unless we find ourselves with an unavoidable need and no other choice, let’s not use that for any other purposes, for now, please?
___
[1]: My view and understanding is that identity cannot be “provided” - those words simply don’t make sense together. Unless if we’re talking about impersonation and skip the “credentials” for brevity, and then it’s not our identity but someone else’s (even if created specially for us). Of course, I could be wrong.
As for your question: sadly, I don’t have a solution for either. I wish I would. I think ML-based approaches seem to show good promise for spam detection, though? I haven’t looked under the hood any recently, but purely anecdotally, almost every time I upgrade my mail system and antispam has something new ML-based, I’m getting a lot less junk. As for the sybils… I don’t think it’s an issue per se - an ability to have alter egos is not a clear negative. And then it must depends on the exact context. Government elections is one thing, online content popularity measurement is entirely different. Not sure it’s meaningful to envision any universal solutions - they tend to have too many side effects, and usually of undesirable nature.
Email is still a protocol, and the thing that ATProto is doing causes as many problems as it purports to solve.
Mostly because "decentralized identity" is still "identity." And the safest way to do identity is to have it be destructable and remakable on the fly.
It might be the safest, but it defeats lot of the purpose of identity. There is a reason it is a hassle to change your email address... so many services are tied to that identity. You can change it, but you have to change every service that is relying on it as your identity, and you still have to own your old email so you can prove to the service that you are the same person.
I am not sure how you could ever avoid this problem? The purpose of an identity is to be able to tell that one request is made by the same person who made a previous request... persistence is a requirement.
Identity is always hard, and I strongly doubt there is any great way that makes it "easier" and still safe.
Aka, yes please kill passkeys, or at least be super upfront and informative.
"When you use passkeys, you are giving your keys to Apple or Google, and they cannot guarantee safety."
> "When you use passkeys, you are giving your keys to Apple or Google, and they cannot guarantee safety."
Not true with hardware passkeys, which also add a true second factor. Central passkeys are a problem, though.
To go on a tangent - I think that more people having personal public key pairs (via crypto) than ever is actually a positive direction. Atprotocol is another big player in identity at the moment, just as long as "can't be evil" mechanisms are kept alive and have good UX.
the intersection of people who aren't sheepish normies, get stuff done and also chat, is almost entirely covered by forums and IRC, as they had been for decades and they don't really have a need for anything flashier
if you want for some reason to have a self-hosted chat of sorts, imo your best bet is to install it on your own website next to your own forum/BBS and pretend it's not really an open messaging service so authorities don't end up coming to you asking to id your users (ideally you'd be moving your server to a non-authoritarian jurisdiction outside of China, UK, Iran, North Korea, possibly the USA at this rate, etc)
basically stay below the radar and enforce compulsory anonymity in your spaces to be on the safe side, that way you only need to gatekeep registrations and the occasional ban and moderation
I've also tried the plain old Asterisk with deskphones and softphone, a nice journey, but not something that could possibly succeed.
I line Nostr, but... So far it lack way to much clients to be used on scale, meaning the reasoning that a scrap of text is the center of our information/communication needs is very nice. But... Most clients are or buggy and limited or monsters not much less buggy. Long-form notes to makes personal blogs seems to be neglected, emails equivalent seems to be just an unfinished and abandoned experiment. The media part is still to be seen in realist usage terms.
So well... The problem of protocols is that lacking a decently feature complete simple app, easy to deploy from go install/pip install/cargo build without a gazillion of deps and different services, easy to package for distro the result is a messy ecosystem only some devs explore to explore, not really to use "in production" and there is so no option to really grow big.
And anyway, people like services. People want services. And they're right to do so! Services are good and necessary. We just need to avoid using services that become corrupted by dark patterns, and to do that it will help if we can put guardrails in place to regulate services.
The example in the article about "moving your email from gmail" is nonsensical. Yes, you can have another email that can still (in theory) email all the same people. But if you don't actually have all your old emails, that will be pretty cold comfort. Not to mention that you may have difficulty convincing those people that you're the same person whose gmail got nuked. And that's assuming whatever new email provider you get is big enough that it won't be autoblocked by gmail, preventing all your old gmail friends from even seeing your emails.
Email being a protocol doesn't really help much here. What we need is to simply do regular "maintenance" of the tech world to ensure that individual providers do not become too big and powerful.
Interoperability has always been paramount, but gets so easily forgotten.
Understandably the attacks put a sour taste in the mouth of many FOSS projects; especially those with corporate sponsors or aspects. 20 years of working fine not considered. But now we're seeing you get the same downsides with corporate services but none of the ability to effortlessly move. Going to Discord has been shown to have been a long term mistake even if it seemed nice in the short term.
None of this could happen with a protocol. You cannot require age
verification on IRC, XMPP, ActivityPub, Nostr, or Matrix, because there is no
single entity to compel. Each server operator makes their own decisions. A
government would need to individually pressure thousands of independent
operators across dozens of jurisdictions, which is a legislative and
enforcement impossibility. And even if one server complied, users would
simply move to another.
Since the spec includes identity, content (in multiple formats), and authenticity/integrity, this makes it superior to nearly all alternatives for offline use. Once you know someone’s key, you can verify that content comes from them, however you manage to obtain that content.
To connect you would run a script called gridstart that mounted the remote resources in that windows namespace then start a sub rio running gridchat, Acme (text editor), page (document viewer) and the mothra web browser that only supports basic html, no js, css, etc. Gridchat was nothing special, it was pretty much IRC with a slight twist. It consisted of a shared buffer living on a 9p message queue server which everyone's client, an rc script, read and wrote to. Some users wrote their own chat client scripts and of course you could completely change how the grid behaved on your end - it was completely within your power to arrange those resources as you saw fit.
The idea was the plumber in that namespace lets a user plumb a message to everyones clients who were listening on that shared plumber. So if you plumbed a url in gridchat it would load in everyone's browser. you could upload an image file to the griddisk, plumb it and it opens in everyones page. Same with source code but Acme would open the file. It was like a primitive slack or discord where you could technically send images, gifs and urls.
All of it was built on Plan 9 tooling using the native 9P protocol and wired up using rc scripts, all of which is available out of the box. I think the only non-standard Plan 9 tool was was the general purpose message queue 9P server that happened to be the perfect tool to host the gridchat buffer. Sadly, Mycroftiv passed away and 9ants is no more but gridchat lives on sans the shared plumber stuff.
It was all about the protocol: 9P. Everything used the same 9P shaped plug and socket and the client was built up from base tooling. You as a client had complete control over the client portion. This was probably the best example of "protocols, not services" that I have ever seen.
If 10% of hosts (maybe even less) are penalized, the rest will likely start complying. much like self managed compliance of thousands of companies. A protocol is only as good as the entities that participate in the community using it.
LLMs are making software easier to write and releases are increasing. The app stores that were not seeing an uptick last year are now showing the uptick in releases. It is happening.
This means software will be more competitive and lower margin. This sounds like doom but it's actually great. Great for consumers. Great for indie devs that want to compete against big companies. Their margin is your opportunity.
Meanwhile, the kinds of early adopters that you're looking for are very conscious of enshitification and lock-in. So the best way to reach them and get talked about is through making software that the big VC-backed companies would never write.
The winners will be one-man companies who understand and respect their customer. Open protocols show your users respect and could be a great differentiator.
Yeah, I also love my data uploaded to public Firebase buckets.
Vibe coding is not the answer to every problem.
I wanted to code in a 'real' language like C. I didn't respect the web technologies. I do now.
It's disservice to yourself to not use the tools available to accomplish your goals. I know the anti-AI sentiment is hot and sometimes for good reason. But there's value here, too.
As for open protocols, there are really two paths. You follow an open protocol that is already out there. Or you can, if you already have some success in your niche, open your SaaS up to be communicated with which can be the start of an open protocol.
With my own software, I'm making it easy for a user's LLM to interact with my software while not providing the AI tool myself. Through a copy markdown button that instructs the LLM how.
This isn't quite an open protocol but has some of the properties of them. It allows people to build integrations ad-hoc without much work. It is on their terms, not mine.
Right now, this seems to be the most ergonomic and transparent way to get integration that allows the user to be in control. And, for my own consumer perspective, the way I hope things go.
Now is a terrific time to be the change you want to see in the world.
What about applications? federations, or better: relays, would put an end to censorship. Encryption would put an end to surveillance. Cryptographic signing would improve authentication and security at wide as there would be no stored passwords to leak.
Until then, "protocols not services" will remain a privilege for the technical elite.
The identity point in the discussion is spot on. The missing piece in most protocol-first architectures is a portable identity layer that doesn't just recreate the service dependency at a different level. DIDs and Verifiable Credentials are trying to solve this but adoption is glacial because there's no compelling consumer use case yet — it's all enterprise compliance stuff.
The XMPP vs Matrix debate is interesting but somewhat misses the point. Both protocols work. The reason Discord won isn't protocol superiority — it's that they solved the 'empty room' problem by piggy-backing on gaming communities that already had social graphs. Protocol design is necessary but not sufficient; you also need a migration path that doesn't require everyone to switch simultaneously.
Use Workflows and Policies, not Agents.
Agents is what they called programs in the Matrix. They were not helpful. Trusting AI Agents is dumb. And Agents can go rogue.
Could workloads really be broken up and distributed like this among many peer machines?