Comment by satvikpendem | Hacker News Reader

Comment by satvikpendem | Hacker News Reader

Parent Full thread

See also, Autosurgeon (with a 0.3.0 release today), which is a higher level API on top of Automerge for Rust:

I'm building a mobile app with a server backend, and I was looking for resources to build them in an offline-first way (since unlike on the browser, people expect to use apps offline, if they can, such as fitness or habit trackers).

I found the concept of conflict-free replicated data types (CRDTS) interesting as it allows you to have fully offline experiences while also having a conflict-free syncing experience. I was looking for some good libraries and came across automerge [0] and yrs [1], but both had some rough APIs as they're primarily low-level Rust libraries that are wrapped by higher-level TypeScript APIs.

Autosurgeon wraps the low-level API of automerge to make it much more ergonomic, closer to the TypeScript experience, but in Rust of course. You can for example use `struct`s which autosurgeon will serialize and deserialize automatically, which is not present in base automerge, which focuses more on string keys and arbitrary values.

I am planning on using this together with Flutter and flutter_rust_bridge [2] in order to use this same Rust library everywhere. In this case, the server just becomes another (albeit more privileged) client.

[0] https://github.com/automerge/automerge-rs

[1] https://github.com/y-crdt/y-crdt

[2] https://github.com/fzyzcjy/flutter_rust_bridge

Be careful when using CRDTs. Having no conflicts does not mean the end result is correct. In many cases you essentially converge to last-write-wins with respect to the Lamport clock.

Generally agree with this. There is no magic solution to resolving conflicts in multi master systems (despite what some database marketing may imply). CRDTs are predictable but they are 'dumb' in how they automatically merge. Make sure the outputs are likely to make sense for your problem domain.

The fundamental thing is that no merge or consensus algorithm can somehow telepathically know the real world intent of its users.

CRDTs can best be thought of as a way to eliminate spurious and false conflicts, leaving only real errors. Without them anyone who has ever coded a data merge knows you tend to get a ton of noise.

So basically you have reduced the problem surface area.

Can you expand on "spurious and false conflicts" here?

Not the OP, but I'm guessing he's referring to, for example, two users each correcting a typo in a different location in the document. From the perspective of the text CRDT, there's no conflict, and users are likely to agree. Raising a "file edited simultaneously, choose which version to use" error would be a "spurious and false conflict" in this sense.

Note that from a different user perspective, say a code document, such a conflict is actually correct and desired. So it's all about context.

I prefer to think of Automerge as a form of version control: because the full history is retained, if you don't like the merge you can decide what you want to do instead.

In automerge's (and usually any CRDT implementation's) case, if it encounters a merge conflict, it will allow you to handle it with a custom merge function. So it's not necessarily that CRDTs are truly "conflict-free," just that it will merge correctly in all other cases than editing the same value at the same time.

While this is true, the base "text" CRDT generally does the right thing for user documents, and conflicts are generally handled reasonably (though it's fair to say a bad conflict would not be automatically resolved "correctly"). Yjs (not Automerge) also has an XML CRDT, which extends the text CRDT to always have correct XML syntax (although again, which text falls into the <em> and which text falls outside of it may not be "correct" in the case of a conflict).

Is it possible to have a selectable roll-back/diff feature such that if the sync goes through - the originals on both sides have a 'backup'/source-of-truth option such that you can revert easily?

Yes, the full history is always retained

Autosurgeon repo: https://github.com/automerge/autosurgeon

Thanks for the links, this is pretty interesting stuff. Just a quick note: it's Conflict Free Replicated Data Types - not Relational.

Yep you're right, fixed.

This is neither here nor there, but I've always preferred "convergent replicated data-types", as there is some confusion about what the "true acronym" was intended to be.

Another thing to keep in mind is that if you want the data to be end-to-end encrypted, then you need both devices to be online at the same time to sync with Automerge.

Reply on news.ycombinator.com