Automerge: A library of data structures for building collaborative applications

There's also Microsoft's Fluid Framework and Azure Fluid Relay which is powering their O365/SharePoint product.

In a recent (abandoned) project, I used Reflect: https://reflect.net/

It was by far the most developer-friendly experience I've had trying to implement collaborative editing. The one thing it didn't have that YJS did was built-in undo/redo.

yJS has the webrtc adapter and appears to still win out in the edit benchmarks. I’ve used yJS in two projects: once just for presence chat and syncing menus for a coaching site and once in a overlay graphics app for a livestream

Biggest problem with yJS for me has been the ergonomics when I use it with React. There’s a third party project called synced store that I used for the stream overlay but it has some strange behavior.

With first party support with React in automerge I think it’s worth a shot for my rewrite

I’m quite familiar with both, having spent some time building a crdt library of my own. The authors of both projects are lovely humans. There are quite a lot of small differences that might matter to some people:

- Yjs is mostly made by a single author (Kevin Jahns). It does not store the full document history, but it does support arbitrarily many checkpoints which you can rewind a document to. Yjs is written in JavaScript. There’s a rust rewrite (Yrs) but it’s significantly slower than the JavaScript version for some reason. (5-10x slower last I checked).

- Automerge was started by Martin Kleppmann, Cambridge professor and author of Designing Data Intensive Applications. They have some funding now and as I understand it there are people working on it full time. To me it feels a bit more researchy - for example the team has been working on Byzantine fault tolerance features, rich text and other interesting but novel stuff. These days it’s written in rust, with wasm builds for the web. Automerge stores the entire history of a document, so unlike Yjs, deleted items are stored forever - with the costs and benefits that brings. Automerge is also significantly slower and less memory efficient than Yjs for large text documents. (It takes ~10 seconds & 200mb of ram to load a 100 page document in my tests.) I’m assured the team is working on optimisations; which is good because I would very much like to see more attention in that area.

They’re both good projects, but honestly both could use a lot of love. I’d love to have a “SQLite of local first software”. I think we’re close, but not quite there yet.

(There are some much faster test based CRDTs around if that’s your jam. Aside from my own work, Cola is also a very impressive and clean - and orders of magnitude faster than Yjs and automerge.)

I’ve been working on a personal finance tracker that uses automerge as the primary backing store.

One trick we’ve been pulling is tailing the automerge contents into a sqlite db in-browser for more complex querying.

(some notes on how/why here: https://tender.run/blog/tender-and-crdts)

Many projects use Yjs for its collaborative rich-text editing (e.g. Linear: https://x.com/artman/status/1733419888654291102). Yjs makes this easy by providing "bindings" to various rich-text editor GUIs, which sync between the editor's internal data structures and Yjs - something that involves a lot of detail work. Automerge's rich-text support is more recent (~last year), and so far they only have one editor binding (ProseMirror), so Yjs is naturally more popular here.

For non-text collaboration, there is a more crowded "market", because it is an easier problem to solve - at least when your app has a central server. Tools range from hosted platforms like Firebase RTDB to DIY solutions like Figma's (https://www.figma.com/blog/how-figmas-multiplayer-technology...). Meanwhile, Automerge's target niche is decentralized collaborative apps, which are rarer.

Big discussion of local-first a few weeks ago: https://news.ycombinator.com/item?id=40786425

A number of past discussions too: https://hn.algolia.com/?q=local+first

I just built a local-first app called "cluttr".

Uses OPFS + WASM to run SQLite and host files. I've had a pretty healthy response to it from HN so I plan to make a desktop app + add PeerJS for file sharing.

Client first also supports users in poor networked areas.

Because you haven't created it yet.

Why a language and not a framework? I mean some langs even implement half their functionality in std libraries rather than part of the lang (as a preference actually).

I see their being overhead and what not that would make it very domain specific and less appealing for anything that doesn't need collaboration.

I can't picture how that would work. While collaborative features require that some objects are shared and synchronized, efficiency and programmer sanity rely on the fact that some objects are not. If synchronization is opt-in, how would a language integrate it any more fundamentally than a library can?

Also, CRDT's don't provide synchronization for free. They ensure that all concurrent modifications will be merged somehow. If the data being synchronized has any structure, it requires careful CRDT-aware data model design to ensure the merging is semantically reasonable (or that, in the worst case, incompatible changes produce a detectably broken state).

Why would you even want to start an entire new language with no tools, no debugging, no libraries, no ides and no optimizations just so you could make some data structures that could be made with C++ classes?

It'd be great, but I think that we are not ready for this yet. Think on how long we needed to make the idea of having good tooling, cross-compiling support, testing or debugging support a requirement.

A library or even built-in language support for distributed data structures will take a decade or two to get to the point of proving a set of features to be truly helpful and good/necessary to have as a library or maybe even as a built-in feature.

BTW, we don't even have quickcheck yet, since the generation and proof reduction quality isn't great across implementations :/

security is a big reason to go local first in general (not necessarily with crdts but they are a strong option), less data over the wire to central servers, you can avoid even needing to know about user data if it stays on their device(s)

collaboration software benefits a lot from this approach because people can just use it and know it won’t get out of sync

Also, anything where you might need data offline at a specific moment. Like if the internet goes down but you still need the thing to show to the guy at the place, you’ll be glad if they used crdts because you’ll automatically have a copy

Also, cheapskates who don’t want to pay cloud bills

Construction industry is a good one: often you are working in a remote area or have cut power (so no WiFi) and having access to your data is key. That’s why there is so much printing still going on- paper works offline.

Anyone using their laptops or phones on public or private transport, or when you have sporadic or unreliable internet.

People who are experiencing a natural or political disaster which partitions the internet.

Found it https://github.com/automerge/automerge/blob/b8f88cacc6f46f92...

Vector clock. Why not a Hybrid Logical Clock?