Hacker News Reader

> Native apps dont deal with the same issues around storage,

This is a synthetic problem on iOS, created by apple to push users away from webapps. It’s not only safari, apple has completely blocked any browser engines other than their own.

Yes, a lot of commenters justify this by saying they want the web to be documents, and everything else native apps, that every page doesn’t have to be an SPA.

For me it’s the opposite - I really don’t want to install another invasive app, just gimme a webapp. But sadly, apple crippled both storage and notifications enough to force developers AND users to use their appstore.

This is lock-in, against the interest of their users, for their marketshare. The exact same tactics that MS was crucified for.

Your comment about an offline first native app being magical has just made me consider… are we in an age where it’s more common for developers to not have lived thru pc apps pre-internet/early internet? What you describe used to be the norm.

Yeah,I'm trying to implement an electron offline first app that syncs, there seems to readymade solution.

Stuff like https://github.com/aerogear/offix seem to be in the right direction of what I'm looking for but not nearly mature enough.

I don't want to pu to much effort on the app so I would like something more or less ready made, preferably with graphql apis.

Any suggestions welcome.

I would hesitate to use any client-server setup that isnt also your primary data store. So, if you have an "offline aware, multi-tenant" store like CouchDB but you still need to sync to the primary store which is SQL, then you lose a lot of context and awareness around conflict resolution. If you are going to eventually sync to the other store, I would say only use Couch on the frontend, and do the syncing/resolution from the perspective of the client, since the client knows what it was trying to do and how best to notify on conflict.

The "better" option is to have an offline-aware client-server component (e.g. Realm) which is the primary store as well. This eliminates the sync and so all conflict resolution stays in the same system with well defined semantics.

Thanks for the recommendation, I'm a little bit skeptical of MongoDB, but realm looks nice, Firebase I'm a little reluctant on being so dependant on google, but otherwise looks good.

PouchDB works well as the primary client store and can sync to CouchDB in the cloud. IMHO this is one of the more mature combinations that gives you a ready client side document database that takes care of data replication for you.

It's just unfortunate that running CouchDB in the cloud seems increasingly perilous. Since IBM acquired Cloudant, it's much tougher to run CouchDB "at scale PaaS" on any data center other than IBM's (for obvious reasons), whereas early Cloudant had robust Azure and AWS support for years.

I wish Couchbase were more helpful practically than they try to present themselves theoretically. Even if their products weren't so expensive the impedance mismatch between their version of CouchDB's sync APIs and their own APIs seems to increase by the year, and is pretty noticeable in how different it works from a PouchDB standpoint and how easy it is to break sync. (Impedance mismatches in allowable database names and _id keys are huge on their own that have massive repercussions in application design.)

Even CouchDB is not CouchDB anymore with impedance mismatches of its own between versions. On the one hand it's good that Cloudant upstreamed a lot of their cluster management tools directly into Apache CouchDB 2+ (even as they made their PaaS offering IBM Cloud only [or whatever it's name of the month is]), but huge architectural changes below the covers in CouchDB 3+ start to present their own sync issues akin to but distinct from Couchbase's (and even some of Cloudant's as they seem [?] to be diverging again into their own 2+ fork after all that work upstreaming stuff?).

More than ever, Azure CosmosDB's focus on bare minimum MongoDB capability and not supporting anything like CouchDB sync, despite having close the same raw ingredients (Cosmos' change feed looks a lot more like Couch's, but is just missing a couple subtle things to make it directly and immediately useful for Couch replication) seems like a "CouchDB is dead and not worth supporting" signal from Microsoft.

Unfortunately, I think PouchDB <=> CouchDB replication has past the "mature" point to the "decrepit" and "falling apart" stage, maybe to the point of "evolutionary dead end" if I'm feeling strongly pessimistic enough, and I've been for years trying to figure what to replace it with.

That's a timely response and I too have to noticed that CouchDB may not have the best/compatible future. I have an existing product that depends on PouchDB in the client and the replication.

I'm not really into the managed db service offerings, I like to have control over it that they rarely offer.

My tentative solution for the application is to go with PouchDB on node.js on the backend too.

CouchDB 4 may change a few things in this space. It will be based on top of FoundationDB, which should make it a lot better for running "at scale PaaS".

Though from what I'm reading the FoundationDB internals are adding even more compatibility breaks from databases that work just fine in CouchDB 2 and PouchDB.

It's nice to see "we're trying for internals that make it easier to run it at scale", but it's not really a "at scale PaaS" play if there aren't any vendors on the major clouds that aren't IBM's interested in actually running whatever those internals are as a service.

I share your concerns and avoid it for the same reasons : )

Just giving options and google fodder.

You should try using Firebase. It handles the data-sync for you, and it applies all updates locally first so that your app feels snappy.

Ah yes. Convenience is how Big Tech gets all data to pass through it.

I mentioned Realm because thats what Im familiar with but yeah Firebase is also a good choice if you want a client-server setup.

Firebase works until you want a different conflict resolution then last-write-wins. It is also only offline first if the user is authenticated. Otherwise you need a connection to the servers before using the local state.

Firebase is offline-resistant more so than offline-first I’d argue.

Last write wins is not handling data sync, it's washing your hands of it.

Your users will be frustrated.

The (JS) frontend SDK can be set to either online or offline mode. When in online mode, it seems to insist on downloading full copies of records, even when a version is somewhere in the local cache, slowing things down and incurring cost. Some optimisation could have been applied here, I didn't find anything about it in the docs. Does this match others experience?

> Native apps dont deal with the same issues around storage, and are actually _much_ more performant overall

Its funny you have to say this - there's a whole generation of programmers who just think web first.

> At the end of the day, offline-first is a whole bunch of caching, and so you end up needing to deep dive into cache invalidation strategies which we all know is a hairy problem.

Looks like you're not truly getting the concept of Offline-first apps then. You don't have or need a cache. You have a local database on the device that syncs up with the server, if online.

Conflicts can occur, but modeling data well with a CouchDB/PouchDB setup (i.e. prevent conflicts by not modifying docs all the time, rahter create new ones) + having a simple time-stamp based heuristic can already be sufficient. Otherwise, using CRDT is an option.

> You don't have or need a cache. You have a local database on the device that syncs up with the server, if online.

The source of truth is the server. Everything else (i.e. the local copy) is a snapshot of that, aka a cache. Its just that offline-first is _always_ a cache-first read, so you seem to think that this makes it not a cache any more, but a regular data store.

CRDT is really a multi-master model with eventual consistency. It's not merely a cache but also operates as a master and source of truth. With a well-built CRDT you can also skip the server and sync client-to-client where both have master copies.

The tricky part is conflict resolution.

Right, the peer-to-peer or anything with a concept of multi-master / majority is a different class of problem. I am speaking only to traditional client-server scenarios.

> so you seem to think that this makes it not a cache any more, but a regular data store.

Yes. In an true offline-first approach with CouchDB/PouchDB, the client-side database can be considered to be main data store and the Server-Side DB could just be a backup. Or it might not be needed at all/ might only be used to migrate from one device to another.

I'd say whether it's a master-slave or Multi-master model depends on the conflict resolution strategy.

That just sounds like last write wins - the first client to sync with the server is now the source of truth and other clients that were working on the same thing will get clobbered.