Hacker News Reader

Comment by hugodutka | Hacker News Reader

We've been using https://github.com/electric-sql/electric for real-time sync for the past month or so and it's been great. Rather than make you think about CRDTs explicitly, Electric syncs an in-browser sqlite db (WASM powered) with a central postgres instance. As a developer, you get local-first performance and real-time sync between users. And it's actually faster to ship an application without writing any APIs and just using the database directly. Only downside is Electric is immature and we often run into bugs, but as a startup we're willing to deal with it in exchange for shipping faster.

View on HN

I've been wondering how well Electric's been working for people ever since I heard about it; good to hear that it's been useful for you.

Couple of questions:

- How big is the WASM blob that you need to ship for in-browser SQLite? Have you had any noticable issues from shipping a large payload to the browser?

- What are you using to persist the SQLite database on clients? Have you been using the Origin Private File System?

This is the WASM blob and it's 1.1 MB uncompressed. https://github.com/rhashimoto/wa-sqlite/blob/master/dist/wa-.... No issues - it's cached by cloudflare.

We're using IndexedDB. Here's a writeup on alternatives https://github.com/rhashimoto/wa-sqlite/issues/85 and a benchmark https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html

Gotcha, interesting. 1.1 MB isn't too bad, especially with Cloudflare providing a local PoP. And if this is for Hocus, I'm guessing your frontend isn't used much on mobile devices with iffy connections.

That writeup on different SQLite VFS's for in-browser use is helpful, thanks for linking that.

How do you handle migrations?

Every postgres migration is done through an Electric proxy and it converts it into a corresponding sqlite migration that it can apply later on the client. In case of a migration that would be somehow breaking you can also drop the client-side sqlite database and resync state from postgres.

The docs say Electric propagates migrations (DDL) on Postgres synced tables to their “satellite” clients

What kinds of bugs have you run into? Any large-scale corruption of data at rest?

We have run into queries that corrupted the database client-side, but fortunately that doesn't propagate into postgres itself. In that case we had to drop the client-side db and resync from a clean state.

The corruption was also caught by sqlite itself - it threw a "malformed disk image" error and stopped responding to any further queries.

Also bugs around syncing some kinds of data - one bug that's already been fixed was that floats without decimal points would not get synced. https://github.com/electric-sql/electric/issues/506

In general electric's team is very responsive and fixes bugs when we bring them up.

SQLite had 2 bugs[1] where batch atomic writes would corrupt your DB if you used IndexedDB to back your VFS. It has been patched in SQLite so rolling a new electric release that pulls in the latest SQLite build should fix that.

[1] - https://github.com/vlcn-io/js/issues/31#issuecomment-1785296...

Any idea on what the root cause of the sqlite corruption was? There's some discussion on the SQLite forums about corruption with wasm (I've encountered it myself on a personal project), but from what I understand no one has identified a cause yet.

How do you deal with shapes and permissions not being available yet?

There's a workaround - if a table has an "electric_user_id" column then a user with that id (based on their JWT) can only read rows which have the same id. It's basic but it works for us. https://electric-sql.com/docs/reference/roadmap#shapes

How are conflicts resolved?

With something called Rich-CRDTs - they were invented by Electric's CTO. They have a section in the docs and some blog posts dedicated to it: https://electric-sql.com/docs/reference/consistency#rich-crd...

Reply on news.ycombinator.com