Automerge: a new foundation for collaboration software [video] | Hacker News Reader

Automerge: a new foundation for collaboration software [video]

youtube.com

68 pointsmr_tyzik29 comments

Relevant github link:

Thanks for that. I wish they were a little more honest at the top when they claim there are no merge conflicts. Yes, they can automatically merge, but some cases will inevitably produce conflict objects and you are going to have to handle them in some way, or indicate to users that some of their edits have been lost.

The downside of anything like this is that for it to work well everything change must go through this framework because the fine grained change information is needed to effectively replay changes. That can make it hard to use existing domain libraries which might require a round trip data conversion as it may not be possible to differentiate between an insertion plus a deletion versus a complex modification.

How Yjs works from the inside out https://www.youtube.com/watch?v=0l5XgnQ6rB4

I really love Automerge, had some fun in the last months with building some small experiment. Some major problem I see currently is the lacking ecosystem.

I had to roll my own persistence implementation on top of IndexedDB and I still haven’t touched the backend synchronization part.

I would love to see Y.js or Automerge more throughly supported by a wider ecosystem of elements that make building a production app easier.

I agree but they do seem quite focused and have made a lot of headway over the last year or so when I first started following the project. I think switching to a rust backend was the right move and will open the project up to a lot more platforms than other implementations.

Looks like both Y.js or Automerge communities are busy with benchmarks

Y.js has pretty good integrations with editors. What would you wanted to see in particular?

BTW Rust port of Y.js might be start of something rather wild. :)

Are you looking for https://github.com/rust-crdt/rust-crdt and https://github.com/libp2p/rust-libp2p?

Main takeaways from toying with both Yjs and Automerge:

1. Extremely difficult to build backend in other programming languages than Nodejs

Rust implementations https://github.com/automerge/automerge-rs https://github.com/yjs/y-crdt

You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs

C-binding, FFI, etc

2. Both communities are great. Before committing recommend to go through issues and discussions

Yjs forum https://discuss.yjs.dev/

Automerge Slack https://join.slack.com/t/automerge/shared_invite/zt-e4p3760n...

3. Watch out implementations of underline libraries. Trace lib0 libraries usage and internals in Yjs for example

https://github.com/yjs/yjs/blob/7bd764fba73fecb8bacd1eb82a4b... https://github.com/dmonad/lib0

JavaScript engines use UTF-16 encoding. Golang (my main backend language) is using UTF-8 ... reimplementing Yjs code in Golang with algorithms and optimization and futher scaling might become impossible for small startups.

Build parallels with Automerge

4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines

BitPhinix seems making a progress on updated example https://github.com/BitPhinix/slate-yjs/tree/next

However there is a surprise waiting for you on backend https://github.com/BitPhinix/slate-yjs/blob/next/examples/ba...

See "@hocuspocus/server" https://tiptap.dev/hocuspocus/

5. There's ProseMirror editor for collaborative editing. However you might not like its internals compare to Slatejs (which is far from perfect and lot of people rely on hard forks)

Feel free to ping me on Telegram @reactima if you are committed to build something similar to Notion.so or thinking to add collaborating editing with Slatejs and Reactjs. I'm really curious how other people plans to live with Yjs and Automerge impressive foundation.

I’m a bit biased, but I think 2022 will be the year of Y.js.

What we’ve done or set up for the next year:

* The Y-Collective to fund more and more related projects, like slate-yjs: https://opencollective.com/y-collective

* The Hocuspocus backend you've discovered, which will become public in April or so

* A cloud offering we’re bootstrapping to hopefully get even more money into the ecosystem

* Y.js core support for the Tiptap editor (900k downloads/month), with a lot more advanced features coming next year

* More and more frontend libraries, like SyncedStore which was on the frontpage a few days ago: https://syncedstore.org/docs/

* The Y.js Rust port: https://github.com/yjs/y-crdt

Exciting times!

CRDTs (and Y.js especially) will deliver what Google Wave promised. Not as a single unifying UI for communication, rather all existing apps can get collaborative power where it makes sense. And that is a game changer for me – as a user and developer. Exciting times indeed!

How do the performance and features of Yjs compare to Automerge?

https://github.com/dmonad/crdt-benchmarks

Not completely up to date I believe but it shows that Yjs significantly outperforms automerge.

It will depend on your use cases. Better to check updated roadmaps too. The final gap won’t be too big IMHO

I agree with Hans, we are going to see an explosion of collaborative tools next year. And Yjs is best placed to enable them.

The ecosystem around Yjs with the Y-collective is so impressive.

I’m biased into Yjs too and bet on it. Yjs Rust port and other simply not ready for the prime and won’t be for a while. Bindings and unsafe Rust is the road to hell

> 4. Rich editing similar to Google Doc is very very complicated subject with lot of landmines

They (Ink & Switch) are actually working on that too, see https://www.inkandswitch.com/peritext/

Wow - you weren't kidding - the rust port is total trash. What is even going on there?

I'm one of the maintainers of this project: we're happy to hear some constructive feedback and proposals for improvements, so please don't hesitate to speak your mind at github issues: https://github.com/yjs/y-crdt/issues .

The internals are still very hot and in a state of flux, as we 1st decided to go with porting the Yjs, then leave cleaning and optimizations for 2nd step after we have something, that's compatible with existing Yjs behavior.

> You will cry looking at source code https://github.com/yjs/y-crdt/blob/main/yffi/src/lib.rs

Am I missing something? You linked a file that is inherently going to be "un-Rusty" because it's meant to be an FFI shim for _non-Rust_ languages to use through a common C API that this file exposes.

One sort of issue is that this only directly supports documents/DAGs, so representing many-to-many relationships is a bit iffy. This is similar to how Rust can't directly represent cyclic data structures.

It's not a huge deal, because working with indices works well enough, but it is a bit annoying.

what problem does this solve?

MeteorJS solved this 12 years ago...but good to see more thoughts being put into this.

The key feature of automerge is that it's decentralized like git. Two people could work asynchronously and only occasionally merge their results.

It did not at all. It solved the problem of how to reflect a source of truth (the server) in somewhat real-time on the client through subscriptions. Also made some attempt to have edits that were persisted to client-side mini-mongo pushed to the server but this was best effort at most.

CRDTs are on a completely different level. They provide deterministic conflict resolution strategies through either/or causal and physical timestamps to ensure it's always safe to persist a write locally and later sync it with the server or not even have a server and sync directly which other peers.

I understand the difference.

However, the local mini-mongo is acting a local-state and the JSON/JS all the way is solving the marshalling/ORM issue. I understand it is a centralized way of dealing with this, however, it was operating on the same problem space. Furthermore, automerge can be used as the server, which bring it closer to what Meteor did. I was hinting at the overlap in the problem space, but I should have phrased it better. Anyway, I'm keep to try automerge.

You can't just use automerge on the server, it must be used on the client and additionally on the server (or a more cut down version that just replicates deltas between peers).

If you are going to try out automerge you should also take a look at y.js and diamond-types.

Yes, I get that part. I'll take a look at y.js and diamond-types, thank you!

I get it, it is different Thanks for correction.

Reply on news.ycombinator.com