Ask HN: P2P Databases? | Hacker News Reader

Ask HN: P2P Databases?

I had an idea for a project that would be best served by publishing a database that all the clients would sync locally and query against directly.

I've looked around a bit and I've found BigchainDB, however it's a quite heavy load. I want to update the authoritative database and have it generate deltas that are synced to the clients via a p2p like network, rather then having anything 'mined'.

Is anyone aware of anything existing that would be of use?

View on HN

Couchbase Lite offers delta-sync and P2P replication over any transport. Has client libraries for all mobile OSes, Java, and recently C.

CBL is backed largely by Sqlite with features layered on top. Local query support, server replication and a bunch of other nice features useful for an offline/P2p database.

Disclaimer: Employee working on the server-side Sync Gateway component for Couchbase Mobile.

This is super cool. Thanks for letting me know about it!

https://github.com/backtrace-labs/verneuil/ is one way to address the diffing / read replica part of the problem. I believe it's compatible with gossipping: most of the data is in small content-addressed chunks, with small manifests that tell clients what chunks to fetch and how to reassemble them to recreate a sqlite database. There's already client-side caching to persistent storage, and chunks can be fetched on demand.

Sharing replication data P2P, while retaining the simplicity of a single authoritative writer per database, is explicitly part of the project's long-term goals!

This is super cool. Thank you.

Git. Seriously, git is, depending on your data model, possibly the best delta compression and sync model you can find. There pitfalls and sync issues to consider, but if it's just going to be published by one authority, git is a great solution.

This is an interesting idea. I wanted to use p2p distribution to save bandwidth costs, but maybe I have to re-evaluate with git in mind.

Thanks for the idea!

Git can be immensely P2P, though there's a coordination problem that would be involved. There was an attempt at "Gittorrent" at one point, Don't know the current status or if there was ever a working prototype.

https://orbitdb.org/

And https://github.com/ipfs/go-ds-crdt

Fascinating. Thank's for letting me know about this!

ipfs is something new to investigate. Have you used this? I'm a little concerned about the data pinning costs.

Ignite works as you describe:

https://ignite.apache.org/

I wouldn't really recommend this approach, I would think more in terms of subscriptions and topics and less of a 'database'.

Thanks for this! I will give my approach more thought with a messaging system model.

Reply on news.ycombinator.com