Electric (Postgres sync engine) beta release

I have followed a lot of these projects because I have clients with persistent state and synced state too, as well as realtime needs. Anyway, this passes the smell test imo. It’s solid engineering and built from first principles, reusing the right pieces. So congrats!

My concern with DB startups is always the business model. There’s a massive tension between open source and a sustainable business which is much more prevalent with deeply technical products. So either it’s too open and dies because aws eats their lunch or it’s too closed to be useful for self-hosting, or too complex to self-host, or priced too high for small players. What’s the strategy to build/grow the business in the short-medium term?

This is really amazing as is their standalone pglite project, which is a WASM port of Postgres. One surprise for me though is that Electric is a read only replica. In particular, “Electric does read-path sync. It syncs data out-of Postgres, into local apps and services. Electric does not do write-path sync. It doesn't provide (or prescribe) a built-in solution for getting data back into Postgres from local apps and services.” https://electric-sql.com/docs/guides/writes

I listened to https://www.localfirst.fm/18 recently from Electric-SQL. One of the things James mentioned was that Electric lets you use commodity CDNs for distributing sync data, which takes the load off your main Postgres and servers.

This seems like a good pattern, but of lower value for a SaaS app with many customers storing private data in your service. This is because the cache hit-rate for any particular company's data would be low. Is this an accurate assessment, or did I misunderstand something?

Informing recent podcast episode with the Electric SQL founder, https://www.localfirst.fm/18.

Related:

Show HN: ElectricSQL, Postgres to SQLite active-active sync for local-first apps - https://news.ycombinator.com/item?id=37584049 - Sept 2023 (171 comments)

Is it fair to say that if you have a high read\low write service, you could use this as a sort of local cache (with the added benefit that it also handles staleness through the sync) on the service itself, to reduce latency from not just calls to Postgres but other caches like Redis?

Just curious. They have Google listed as user. Why would a company like that be interested given that they’ve already any world class db in-house along with the expertise?

The ElectricSQL concept of 'shapes' is interesting. Right now, a shape is just a single table with some column filters and a where clause. The shape is defined on the fly in the query string. Electric does single table sync of those 'shapes' a lot like Supabase realtime.

I work as part of the team building Ably LiveSync, a competitor in this space. Our postgres connector[0] works in the opposite way by taking rows from an 'outbox' table and fanning them out to subscribed clients over websockets in realtime.

Here's the thing; for any meaningfully complicated data in a relational database the data is likely to be normalised across tables with relations, and there are going to be joins. So I'm really curious how people are making single-table-sync work. (Maybe this is where Electric imagine the future of shapes, solving for joins).

In LiveSync we do it the other way around, instead of having a live-query style subscription to a table (like ElectricSQL), we listen for someone writing a row to the 'outbox' table and that row is automatically sent to subscribers. This means that you're writing your denormlised data directly to the outbox and it's being sent on to clients, rather than writing your normalised data to tables but being limited by single table sync. Or worse, your clients having to subscribe to multiple sync streams and trying and stitch the data back together. We opted to have the write-side insert the denormalised data, rather than having the read side have to stitch normalised data back together.

Electric will at some point try and solve multi table sync, which isn't easy given how the Postgres replication protocol works. It's also not easy to imagine how shapes (which are defined on the fly right now in the query string) will adapt to multiple tables. There's going to be a tradeoff between a complex query string trying to stitch the normalised tables back together, or a 'shape' becoming an entity in Electric which would define how to stitch the normalised tables back together (which you would have to CRUD manage, and update in Electric every time your schema changed).

[0]: https://ably.com/docs/livesync/postgres

Eagerly awaiting the React Native compatible version. https://zerosync.dev/ is the main alternative I'm considering.

I may be misunderstanding, but does this boil down to essentially a partial read replica?

I tried using this a bit in an earlier version, when there was an option to have a real SQL client library/ORM, and you declared what part of the schema would exist client side. Is there plans to bring this back?

The part where you could use drizzle client side was really what interested me, I don't want to bother learning another new query language apart from SQL.

Is this basically a read-only partial replica? I wonder if it would be possible to sync the changes back to the upstream.

Caught a few typos on your site as well fyi, https://triplechecker.com/s/153718/electric-sql.com

Hey this is great to see, congrats on the launch!

I've been researching 'local first' solutions like electric recently and tried out powersync, triplit and instant for now. All three of these solve for both reading and writing to databases, with offline support.

Wondering if you have plans to support writes too.

This looks very interesting. Do you have any plans to implement a postgres extension, so that a client postgres can natively sync shapes from an upstream postgres? Very similar to your pglite replica, but for native postgres.

Is there any software that let me make graphical user interface to a connected database, allows me to make data visualizations, all things automatic and interactive? Like a node editor or spreadsheet?

I am going to try this, it looks very interesting

Super excited!