Show HN: SyncedStore CRDT – build multiplayer collaborative apps for React / Vue | Hacker News Reader

Show HN: SyncedStore CRDT – build multiplayer collaborative apps for React / Vue

syncedstore.org

144 pointsYousefED26 comments

Hi HN!

With CRDTs (Conflict-free Replicated Data Types) becoming more mature and getting more attention lately, I decided to dive in and use it as the basis for a new project. I love the technology, and how it enables me to create Google Docs style collaborative apps, that seamlessly sync data across users / devices but also work offline.

However, I also found that it's not yet trivial to build! This is why I started building a library that makes developing multiplayer apps super-simple, with a very minimum API surface. (basically, you can just modify plain javascript objects and changes are shared across users).

I'm now excited to share SyncedStore - and looking forward to your feedback :)

It's designed specifically for React, Vue, Svelte but also works with plain JS. By using Javascript Proxies, the data store looks like plain javascript objects, except that they'll now sync automatically across devices / users!

Would you care to compare this with for example @localfirst/state, automerge and automerge-rs (latter two provide the CRDTs, i.e. similar to Yjs)?

https://github.com/local-first-web/state

https://github.com/automerge/automerge

https://github.com/automerge/automerge-rs

By the way despite that particular repo (@localfirst/state) last being touched 6 months ago, Herb Caudill definitely seems still active in this space (I believe he's been working on other parts of this more recently -- e.g. ideas about authentication), and I think automerge development itself (Martin Kleppmann) is quite active right now leading up to a 1.0 release which seems fairly imminent, for which a lot of fundamental work has been done, also coordinating with automerge-rs.

Hi! Great questions :)

First of all, SyncedStore does not implement any CRDT algorithms. Credits for this go to Yjs [1] (and its author Kevin), which it uses as underlying CRDT.

Yjs and Automerge are (afaik) the two most commonly used CRDT implementations. Both have their pros and cons, but Yjs has focused a lot on performance [2].

Automerge has a bit friendlier "Immer style" [3] API. I'm not too familiar with @localfirst/state, but it seems to add a Redux style API on top of Automerge.

My approach with SyncedStore was really to provide an API on top of Yjs that's as simple as possible to use in React / Vue / Svelte or plain JS app. I.e.: only use a single React Hook to observe changes, and use regular Javascript assigments to update values. The API is inspired mostly by Reactive Programming libraries such as MobX [4] (from the same author as Immer).

Hope you're still following along :) Maybe it helps to compare the TODO-MVC applications, as both SyncedStore (https://github.com/YousefED/SyncedStore/tree/main/examples) and @localfirst/state (https://github.com/local-first-web/state/tree/main/examples/...) have implemented these as examples!

[1]: https://github.com/yjs/yjs [2]: https://github.com/dmonad/crdt-benchmarks [3]: https://github.com/immerjs/immer [4]: https://mobx.js.org/

Wow this is so awesome!

How/where does the data get saved for later? I'm guessing from the example, you don't need a database? E.g. I can just save the data to a document database, and next time someone works on it, just sync with the database? Or do I not need it at all because it's stored on the browser? And how would the information be associated with accounts, if someone wanted that feature?

Also curious how/where the demo site is deployed (guessing Vercel doesn't work since they don't seem to support websockets? I see that Hocuspocus is mentioned — is that required as a backend to make it all work?

Sorry I'm new to all of this and it sounds like magic, but I'd LOVE to learn how to implement this; especially on something like Vercel, if possible

Creator of Hocuspocus [1] (which I’ll open source next year) here. You can use y-websocket [2] aswell. Hocuspocus is a (very nice) wrapper/API around y-websocket written in TypeScript.

https://hocuspocus.dev/

https://github.com/yjs/y-websocket

Thanks! Love the excitement :) SyncedStore supports the same backends as Yjs, here’s an overview: https://syncedstore.org/docs/sync-providers.

You can indeed store updates in a centralized database (wrapped with an API to handle auth / accounts) - this is probably the most common scenario.

But you can also go fully decentralized and use y-indexeddb to store updates in in the browser, and use a different transport to sync updates between users (e.g. webrtc).

—-

The cool thing is that the data model behind CRDTs is very flexible in this regard. All changes are captured in small “updates”, and as long as these updates at some point arrive at your device, your document can be updated with the changes (it’s eventually consistent).

The site runs on github pages without a backend (examples use y-webrtc).

Pretty interesting project. I'm curious what the data structure looks like that is sent over the network for diffs. I found the part about state-vector, but not about the actual data.

Does it diff the JSON string, or the objects/array themselves? Does it serialize the data or leave as plain text?

I ask, because I have a similar setup, that needs to sync dynamic data structures, but also keep the data deltas as small as possible when sharing them as JSON strings to save bandwidth.

It doesn't do any diffing, but keeps track of operations underneath, and syncs those. The gist is something like this: - if you push a string "hello" to an array, it will sync an update [push "hello" to Array x] along with some metadata - if you modify a property "prop" and set it to "value" it will sync an update [set "prop" to "value"] along with some metadata

It will not automatically diff these with the previous value (in a distributed system it's difficult to say what the "previous value" was, but it will sync only those properties that have been changed). For text heavy operations, you can use the Text structure which will sync fine-grained operations ([delete 4 characters at position 2] or [insert "world" at position 5]). This makes it suitable for collaborating on rich text documents (similar to Google Docs), see: https://syncedstore.org/docs/advanced/richtext.

This might also be an interesting resource to learn how the Yjs internals work (SyncedStore builds on top of this): https://github.com/yjs/yjs/blob/main/INTERNALS.md

So cool to see libraries like yours coming out! (am in love with Y.js)

How does the y-websocket handles network failure ?

Does it autoreconnet? On reconnection, does it sync back changes that happened on or since failure ? Can you notify the user when it's happening ?

Yjs author here. On network failure, the client will automatically reconnect and synchronize with the backend. And yes, you can get an event that notifies you about the current connection status. When syncing with the server, you will only exchange the differences that were created while offline.

Thank you, that's an awesome answer in so many ways.

Love this! Thanks for sharing, I have some amazing use cases for this type of work and I can't wait to dig into the code to learn more. The future for CRDT is indeed hot right now and everyone wants it into their app, Google paved the way and now other big names, like Figma, Invision etc are getting on this hard!

What Google work are you referring to?

Google Docs is one of the classic examples

Google Wave specifically, and now anything in Google Docs...

FYI, the examples don't work for me in Brave browser on linux:

React example: "/App.tsx: Cannot read property 'getItem' of null (2:0)"

Vue example: "file:///src/store.ts: Cannot read property 'getItem' of null (1:0)"

Works fine in Chromium and Firefox though, so not sure what's up.

https://brave.com/privacy-updates/7-ephemeral-storage/

Ah. Thanks!

This is awesome to see. I have gotten interested in CRDTs in the last couple years due to them being used in the Elixir/Phoenix ecosystem.

One question though, how do you integrate a backend into your library? I don't see anything mentioned in the docs.

Thanks! As it builds on Yjs (many kudos to Kevin Jahns for creating that awesome project + community), any backend that's compatible with Yjs works.

The demos work based on WebRTC / BroadcastChannel (for the same browser / device), but I've listed different backends you can use here: https://syncedstore.org/docs/sync-providers

Hi! I just saw this reply — could you please put together a quick write up of an example deployment and the steps you took to do that (e.g. for the documentation site), including all the services you use?

I’m building one for Y.js. If you want to take it for a test drive:

$ npx @hocuspocus/cli

Awesome! I assume you’ve been using this in another project?

How did you handle/implement undo-redo?

Yjs author here. Yjs ships a flexible selective undo-redo manager. You can define one (or several) undo managers that listen to changes that you want to be able to undo&redo.

https://docs.yjs.dev/api/undo-manager

How does this compare to other libraries like GUN.js?

Reply on news.ycombinator.com