We've had great success with Replicache+Orama since this was written. We're keen to give Zero a spin once it's a bit more stable.
Triplit has essentially folded as a "company" and become some sort of open-source initiative instead.
InstantDB has matured massively and is definitely worth a look for anyone starting a new project.
Also agreed Triplit's DX is excellent. I'd recommend giving it another look, Triplit's recent 1.0 release has up to 10x performance boost (https://www.triplit.dev/blog/triplit-1.0).
Since your use-case is data in the range of gigabytes, you could consider using duckdb-wasm. However I'm not sure how to best integrate this with collaboration / CRDTs (sqlRooms is also interesting prior art).
We have hundreds of thousands of entities in Replicache, and index them via Orama. We're able to perform full-text search in single-digit ms.
We persist the Orama index as JSON, so computation only happens once per mutation.
Zero (and I believe Replicache as well) layer their own SQL-like semantics on top of an arbitrary KV store, much like the layering of SQLite-over-IndexedDB discussed; like SQLite-over-IndexedDB, I believe they are storing binary byte pages in the underlying KV store and each page contains data for one-or-more Replicache/Zero records. The big difference between SQLite-over-IndexedDB and Zero-over-IndexedDB is that Zero is written with sympathy to IndexedDB's performance characteristics, whereas SQLite is written with sympathy to conventional filesystem performance.
On the subject of "keep whole thing in memory", this is what Zero does for its instant performance, and why they suggest limiting your working set / data desired at app boot to ~40MB, although I can't find a reference for this. Zero is smart though and will pick the 40MB for you though. Hopefully Zero folks come by and corrects me if I'm wrong.
The constructor allows you to pass in any arbitrary KVStore provider, and we happen to use op-sqlite as its performance is exceptional.
There is no "different data layer" per se, just a different storage mechanism.
Replicache also holds a mem cache that is limited to ~50MB if I recall. Our use case is extremely data-heavy, so we might end up never migrating to Zero – who knows.
Perhaps I misunderstood your question, let me know if I can clarify further.
Notion always* has a webview component, even in native apps, but we also have a substantial amount of "true native" Swift/Kotlin. We can't use Replicache/Zero today because our native code and our webview share the SQLite database and both need to be able to read and write the data there; if we use Replicache that would make our persisted data opaque bytes to Swift/Kotlin.
*There's many screens of the Android/iOS app that are entirely native but the editor will probably remain a webview for a while yet.
> Zero (and I believe Replicache as well) layer their own SQL-like semantics on top of an arbitrary KV store, much like the layering of SQLite-over-IndexedDB discussed
Replicache exposes only a kv interface. Zero does expose a SQL-like interface.
> I believe they are storing binary byte pages in the underlying KV store and each page contains data for one-or-more Replicache/Zero records.
The pages are JSON values not binary encoded, but that's an impl detail. At a big picture, you're right that both Replicache and Zero aggregate many values into pages that are stored in IDB (or SQLite in React Native).
> On the subject of "keep whole thing in memory", this is what Zero does for its instant performance, and why they suggest limiting your working set / data desired at app boot to ~40MB, although I can't find a reference for this. Zero is smart though and will pick the 40MB for you though. Hopefully Zero folks come by and corrects me if I'm wrong.
Replicache and Zero are a bit different here. Replicache keeps only up to 64MB in memory. It uses an LRU cache to manage this. The rest is paged in and out of IDB.
This ended up being a really big perf cliff because bigger applications would thrash against this limit.
In Zero, we just keep the entire client datastore in memory. Basically we use IDB/SQLite as a backup/restore target. We don't page in and out of it.
This might sound worse, but the difference is Zero's query-driven sync. Queries automatically fallback to the server and sync. So the whole model is different. You don't sync everything, you just sync what you need. From some upcoming docs:
We're leveraging sync to make high-quality online software. We may come back to offline in the future, but it's not the priority today.
You can read more about our approach to offline here:
https://i.imgur.com/B5iOd9y.png
https://x.com/search?q=aboodman%20local-first%20zero&src=typ...
I’m glad this post made its way to HN, since it allowed me to reflect on how much progress we’ve made since it was written. The blog post doesn’t support comments, so here goes:
> The DX is the worst by quite a margin.
DX is something that we care a lot about. Like it’s a topic in every single planning and strategy session. We’ve made a ton of progress since this was posted, and we have a lot more on this front on our roadmap.
> Not only does it require Postgres-level integration
If you’re looking to stream data from your Postgres database to clients, I’m not aware of any other way to do this but to integrate with Postgres. So I’m not sure why this is framed in a negative light. PowerSync also only requires a user with SELECT privileges, so it’s not an invasive integration.
> it also needs a HA MongoDB cluster
It’s now possible to use Postgres for intermediary storage instead of MongoDB. I actually recall we gave you a shout out in one of our product updates on this :)
> a lot of arcane yaml configuration
We’ve since published a schema for the yaml config - add this to powersync.yaml:
# yaml-language-server: $schema=https://unpkg.com/@powersync/service-schema@latest/json-sche...
> It also required us to completely denormalise our Postgres tables, as relations are not properly supported across the sync buckets.
This is ultimately a function of our decision to build a system that scales well. Having said that, we have plans to address this limitation this year.
> we found horrifying problems like off-by-one bugs in their Drizzle ORM integration
This was fixed around the time of the post, and our Drizzle integration is now in beta i.e. fully supported in production environments.
> queries returning data from local db very slowly (100ms+),
It’s hard to say what was causing this without knowing more, but as mentioned elsewhere in the comments here we’ve since added support for OPFS which provides a big performance boost.
> and long initialisation times (45s+ after login to Marco) with the UI view not updating until the sync fully completed.
We’ve added APIs to solve this - if you want to sync everything up front you can use sync priorities to log the user in quickly and sync the rest in the background [1]. We’ve also implemented many optimizations like transport-level compression, a waaaay faster rust sync client, etc with more to come. Also, using sync streams (available in the next few weeks) [2] you can sync data just in time as the user navigates around the app instead of syncing everything up front.
> No real case studies
It’s often quite a slow process to get case studies published with large companies, but you can see logos of customers using PowerSync on our website. But yes.
[1] https://docs.powersync.com/usage/use-case-examples/prioritiz... [2] https://github.com/powersync-ja/powersync-service/discussion...