ATProto for Distributed Systems Engineers | Hacker News Reader

ATProto for Distributed Systems Engineers

atproto.com

141 pointsLelouBil30 comments

Open articleView on HN

Shameless plug, but if this got you interested, I have a couple of longreads on the topic:

- https://overreacted.io/a-social-filesystem/

- https://overreacted.io/there-are-no-instances-in-atproto/

From the official site, I'd also recommend https://atproto.com/articles/atproto-ethos

I feel like it's the best ATProto architecture explanation on the web.

I hope it will make people who argue that "there are no instances in ATProto" is wrong, understand that it is actually true and not just a naming debate.

Is ATProto federated, P2P, or a protocol flexible enough to build all kinds of things?

Is it opinionated about which type of system to build?

Skimming this page, it looks like ATProto is a schema for the ontology, and despite saying "P2P", it looks built around federation and concerned about scaling infrastructure (rather than scaling message broadcast).

I'm interested in a p2p social network with federated supernodes (p2p first class, federation second class, relays for devices that don't want to join the swarm) for people and agents. Could you easily build that with ATProto, or is it the wrong domain?

I'm interested in true, first-class P2P. Not a bolt-on afterthought. That's the problem with all of these federated services like Bluesky and Mastodon.

A true social protocol should work like email. Any machine on the network should be able to talk it to any other machine. Of course centralization nodes will appear, but they should be optional. Used as aids for user friendliness, anonymity, and firewalling yourself off.

Then maybe what you want is email?

Sprinkled with a little bit of https://delta.chat ?

And potentially a dash of iroh https://news.ycombinator.com/item?id=48542480 https://www.iroh.computer/solutions/delta-chat ?

Delta.chat no longer uses email though, right?

delta.chat is email

https://delta.chat/en/help#can-i-use-a-classic-email-address...

Any PDS can talk to another PDS, yes.

Yes, and is that useful within the structure of the Bluesky ecosystem?

What if PDSes replicated the content of PDSes? Then it'd be available.

They do not directly talk/query each other. But Relays and AppViews allow PDS's to be aggregated to create apps like BlueSky.

I don't think anyone would describe ATProto as p2p

> A true social protocol should work like email. Any machine on the network should be able to talk it to any other machine

You may find littlefedi interesting

https://littleone.littlefedi.social/@stefano/blog/2026/08/07...

Some atproto designers come (somewhat disillusioned) from the p2p world, you can read what they have to say about that here: https://atproto.com/articles/atproto-ethos#peer-to-peer and https://atproto.com/articles/atproto-ethos#synthesis

Mastodon works like email. Users have their data hosted on instances and most big instances can communicate with most other big instances, and all instances blacklist a bunch of other instances for spam or political reasons. Just like e-mail.

ATProto is definitely not P2P and probably not federated. Every user has their own personal storage and gives apps like Bluesky permission to write to them.

They solve different problems and, in theory, you can use ATProto and ActivityPub (Mastodon/Threads/Peertube/etc) together

The article did not make me understand that at all.

> Since we're making a new app, we're going to want two things: an app server (which hosts our API & frontend) and a view server (which collects data from the network for us).

Yes, the user data is decoupled from the apps, but aren't both stored on some kind of instances? Also:

> Why are we listening to the event stream if we're the one making the write? Because we're not the only ones making writes! There are lots of user repos generating events, and lots of apps writing to them!

Who is hosting this event stream? Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?

>Yes, the user data is decoupled from the apps, but aren't both stored on some kind of instances?

There are two main kinds of "nodes" in atproto:

- Hosting aka "personal data servers" (PDS). This is dumb JSON hosting that you can query by HTTP or watch by Websocket. Super cheap to run. They don't talk to each other. You can have one per user, or one per many thousands.

- Apps. These are your normal webapps. (But they ingest data from everyone's hostings.) They also don't talk to each other.

So yes, there are "instances" in the sense of "boxes which run software" but the topology is completely different from Mastodon or such. Data flows from hosting into apps (and then apps write to hosting). There is no hosting-to-hosting or app-to-app connection. Hosting is app-agnostic, and apps are hosting-agnostic.

To make all of this practical, there are things in the middle that make the situation easier for app developers — either relays (which combine event stream from many hostings), or caches like Hubble[1] and Constellation[2] (which let you query the entire network in one request).

>Aren't we making some kind of federated network by deciding on our app server which event streams or which users/apps pushing updates to listen to?

Ideally you would listen to every relevant event from the entire network (and filter out every irrelevant one). It isn't hard today — you can either use an existing relay or run your own for ~$30/month or pool with someone. The discovery mechanism is that (1) a hosting can request any relay to crawl it, and (2) a relay can discover more hostings it hasn't crawled yet by following links — similar to how Google crawls the web.

[1]: https://atproto.com/blog/introducing-hubble-a-public-mirror-...

[2]: https://constellation.microcosm.blue/

Lol reminds me of "ITS SERVERLESS!" Which always cracks me up, as someone who does find serverless useful for very strategic scenarios, naming things as though they don't require a computer somewhere to function has been one of the weirdest trends in tech.

The name "serverless" doesn't mean that there are no servers. It means that, as a developer, you don't need to manage the server. It's abstracted out for you.

I know what it means, I pitched serverless as a new tool to a former employer back nearly a decade ago, I had to thoroughly research it because the manager would of said no if I had no idea how it worked, funnily enough, he said yes before I could finish my entire pitch.

The name still sounds silly.

I don't see the connection. The article is full of talk of servers: app servers, view servers, database servers, event log servers, ...

The article seems to forget about content addressable storage and signing key cryptography and how these tools might make their software more distributed.

my experience with most more-distributed internet-systems is: it's so distributed it's hard to get.

here you have Authenticated Transfer Protocol. you can get data very easily. via any of a huge range of means & sources! we have some examples ones today (direct from a pds, relay, jetstream, appview, new protocols like Atom (atproto over MoQ), new systems like hubble). and in most cases, you can tell that it's from a certain account!

i agree with you, i just think the bases are mostly covered here. i also think that it's a fascinating topic. and yes it matters. an "Atproto for Distributed Systems Engineers" probably should know why atproto has such great addressable data (not hash based though), signed key crypto. but also, if you dig in, you'll find quickly it has a great story here. (although your https://anproto.com/ indicates some clear disagreement about sufficiency.)

and the higher level what it is and how it works here is a much more useful practical guide, for getting started with.

there's a lot of people who focus on distributed, on what that means. what seems a lot missing though is that atproto is already nicely a nicely authenticated (crypto) transfer protocol. and that it's probably the best actually distributed, in terms of data availability, in terms of people trying to get your data.

Yah atproto auths moving data from the pds to the index, similar to like https, but they missed authenticated distributed data storage somewhere along the way. Why is that, I wonder?

Not quite sure what you're getting at with "authenticated distributed data," but records that came from a PDS are signed by the PDS. The signatures are there to enable untrusted mirrors to be used for distribution. There's a downside: you can't really disavow a post. But that doesn't seem to be the purpose of the signatures? It's more of an unfortunate side-effect.

In practice, this can be annoying and there are alternatives for people who write software that doesn't bother to check signatures.

Relying on https alone means you kinda have to get the data from the original website to know that it's authentic, and you can't prove it to anyone else other than by sending them the URL so they can check for themselves. (I mean, technically. In practice, people will trust quotes or screenshots even though they can be faked.)

Yah it clearly has always done this, but why doesn't it support multi-PDS replication?

But you're right that the screenshot proto and the copy and paste proto are quite successful as a distributed system these days.

There's nothing stopping multi-PDS replication right now. It would work fine. No one would know that content is available elsewhere, is kind of the main chicken-and-egg problem.

Hubble is starting to come online, which is a network cache. With excellent storage and runtime characteristics, cheap to run. I'm not sure if this is in the same https://microcosm.blue banner-head, but the same developer, as another incredibly fantastic network service that's open for use, that's open source, that other people can also run themselves. https://atproto.com/blog/introducing-hubble-a-public-mirror-...

I agree that yes the PDS's hopefully start doing more mirroring too. It'd be nice if they would themselves also attest that they have seen it too.

Having the content available everywhere seems like the definition of a distributed system to me. Hashing content is not a discovery issue, one simply iterates over a list of hosts and asks if they have the msgs by their hash.

What would be the upsides and downsides of that? It’s unclear what you’re asserting. Do you just want IPFS?

IPFS already exists. What I want is distributed Bluesky, since it's a product success and I do like most of the proto and the team. Do you want to fix this? I just spoke with Claude and I don't think it'd be too difficult these days for us to fix this in the core product.

Upside: data is always available even if Bluesky's funding is pulled. Downside: we lose deletes.

Shameless plug, we built Chronon to solve a very similar problem. Chronon is now open source (apache 2) and used at Netflix, OpenAI, Airbnb, Stripe etc.

It is used to serve features/context to ML/AI models and rule engines, along with app views.

We are more scale and perf oriented as project given the use-cases and less decentralization oriented.

We also ship with a dataframe / prql like declarative api.

[1] https://chronon.ai

Reply on news.ycombinator.com