P2panda: P2P protocol for secure, energy-efficient local-first web applications | Hacker News Reader

P2panda: P2P protocol for secure, energy-efficient local-first web applications

p2panda.org

217 pointsrapnie37 comments

Oh wow, we didn't expect p2panda to pop up here (I'm adz, one of the core contributors). Let us know if you have any questions :-)

This week we're wrapping up a long development phase for a new "aquadoggo" version (https://github.com/p2panda/aquadoggo/) - our p2panda node reference implementation. The release will come with a dynamic GraphQL query (filtering, cursor pagination, text search and ordering), blob support, a better configuration interface and a full p2p networking stack (QUIC) with NAT traversal techniques and relays based on libp2p.

Next steps in our current funding period is a showcase app for Android (built with Flutter and Rust FFI bindings from our core libraries), a bee sighting database for scientists in the Brazilian Amazon (in collaboration with https://www.meli-bees.org/).

Further down the lane we're continuing our integration of MLS - as some high-level APIs are still missing - and a capability system. We're definitely not out of ideas yet!

This subject is near and dear to me, but after clicking around for a few minutes I was left none-the-wiser about what it actually is. Is it a client? A server? A protocol? Your home page encourages the user to play what looks like a child's game, and your links to "specification" don't actually pull up a specification.

I say this, truly, as constructive criticism: I am probably in your core audience and I bounced off the site.

Thank you for this! We've heard that feedback from other people as well and totally agree. It's true that it is hard to explain what p2panda really is: It is three things at the same time: 1. a protocol specification 2. a reference implementation and 3. an SDK. We can do better at separating these worlds, maybe by giving it different names or a better website. End of our current funding period we dedicated some time for an overhaul.

The specification you find here: https://p2panda.org/specification/

The SDKs you find here: https://p2panda.org/libraries/

The reference node implementation you find here: https://github.com/p2panda/aquadoggo/

For more details what a "client" and a "server" is: https://p2panda.org/learn/networks

Thank you for the response. I will spend some more effort reading your site and related documents. I have a similar problem communicating the value and content of a project myself in the p2p web space. I'm not as far along as y'all. My audience will likely be programmers at first so I'm going to focus on them - and that audience wants to be able to identify what a thing is as fast as possible. I'm a big fan of informal communication, but with this I've found that going out of my comfort zone to be more formal has had a good impact on comprehensibility. Informality is there, but in small doses. I think this helps underscore the seriousness of the endeavor, too, as old-fashioned as that seems.

I'm going to study this project in great detail—it's something I'm very interested in contributing to, or at least building with. I've been hoping for something along these lines since around 2014 and have failed to make sufficient progress with my own attempts, so I'm really happy to see this kind of thing catching on.

A licensing question that pertains to adoption: your license is AGPL, which I'm a fan of except for the practical matter of deploying on Apple's iOS App Store, where Apple's distribution mechanism would make an otherwise well-meaning open source app run afoul of your license.

Do you plan to grant a special license to people building open-source iOS apps?

[edit: typo/clarity]

This looks super cool! I was thinking I couldn't finish my PWA without a database but its much cooler to use this instead I really don't want to store stuff for people who buy my service, I want to sell them support instead of storage.

If that means you will also have a Dart / Flutter package available for development I would be very interested.

We've started implementing one here as part of the "Meli" Android App project: https://github.com/p2panda/meli/ - as soon as we're happy with the API we would factor it out in a separate repository and publish it!

Curios to know why Meli only has Android support? Since Flutter + Rust (FFI) seems a good solution for multi moblie platform support.

Excellent, I did end up taking a look. Hopefully using the FFI approach worked out ok for you folks in the end

Seeing that this was put together by one of the co-authors of the recently released MLS (Message Layer Security) spec and backed by an EU funding grant are both interesting tidbits that got me more interested in this.

This would indeed allow for a very different internet and would provide some viable options untethered to big tech as it exists today.

I case you refer to me as the MLS co-author, I'm not directly affiliated with the p2panda project. I think p2panda uses OpenMLS (OSS implementation of the MLS protocol, https://openmls.tech), hence the connection. I did however exchange with this friendly an motivated team!

Yes, it's been a pleasure to work with OpenMLS, they consulted us with the integration into p2panda! As mentioned in my other comment: while most low-level parts have already been written, we still need to work on our high-level APIs, which should allow for easy integration of data encryption when using our SDKs. We aim at doing this in our next funding period.

Thanks for clarifying that. I’m sorry if I misunderstood but I really appreciate you pointing out those facts.

Wow! This might just be what I've been hoping for, since 2020.

What I hoped for was more of a generic backend for multiplayer games.

But this is even better. It looks like a next-gen bittorrent which natively supports operations?

How did the EU support something like this without worrying about misuse?

It would be fun to build something like a Blender plugin that allows people to cooperate on free 3D models and animations. But it's also unclear whether anyone would be legally liable when something goes wrong.

(Edit: It sounds like we want to avoid legal issues by having everyone run their own local instances. I like that feeling of hosting our own servers. But something just feels odd. Since when are we allowed to be so open? What's the catch?)

Looks interesting, but there doesn’t seem to be any information about what CRDT it’s using, or how it handles concurrent editing at all. The only operation appears to be “set key”, so it looks pretty basic. Is this the target state or is this going to be expanded upon in the future?

https://p2panda.org/specification/data-types/documents

> A document is made up of operations published by one or many authors

> Every operation has a previous field containing a document_view_id which refers to document state at the moment the operation was encoded

> DO3: An operation which refers to the current operation in its previous field MUST be sorted next.

As far as I understand, it would be up to the client to present potential conflicts and won't be a valid document until it has been resolved.

We're using our own CRDT called "Operations" giving us multi-writer conflict-free editing. It's a simple key/value map with a last-write win rule while we keep some sort of vector clock for every write to understand what every peer has seen when they updated the Document.

This might be limiting for some cases, though so far we could model many applications already with such simple CRDT. It is also possible to add your own or already existing CRDT frameworks on top of p2panda.

I wish I could read the specification offline. It's split into a lot of smaller pages.

Should I be this disappointed that there is no C or C++ library?

Well, it says WEB applications, so presumably this is something that will run in a browser, which implies some flavour of Javascript.

Tangentially, I've been interested in this space for a while, and I have a few in-progress projects built on CouchDB. My aim was to offer the offline app for free and monetize through sync / sharing features. This would've mirrored my cost structure (hosting), so I thought it was a pretty fair way to monetize a product.

easy to find from the homepage:

https://p2panda.org/libraries/

> The core library is shipped both as a Rust crate p2panda-rs with WebAssembly bindings and a NPM package p2panda-js with TypeScript definitions running in NodeJS or any modern web browser.

I’ve been waiting for something like this.

There are still many places even in developed countries where a stable network connection isn’t guaranteed

You might like Holochain (https://www.holochain.org/) as well, it's in a similar vein.

I wonder how complimentary this could be for etcd, which tends to not work so great for structured data.

What is the difference compared to IPFS and Torrent?

Is it blockchain?

No. Just old fashioned p2p with signed transaction chains. No consensus, as far as I can tell.

Isn't "consensus" just another word for "(eventual) consistency"? I don't see how you could build a distributed database without tackling the consensus problem.

More information can be found here: https://p2panda.org/specification/replication. The basic data sync is based on an append only log. But I can't find any information on what are the strategies for dealing with conflicts. If any of the contributors see this, could you comment on this please. Thanks!

Check out our section on "Operations", this is the data type we've built on top of the append-only log structure for multi-writer and conflict free data editing: https://p2panda.org/learn/operations

> Isn't "consensus" just another word for "(eventual) consistency"?

In my head, there's a distinction. Consensus can be used for non-commutative operations, whereas signed CRDT chains can operate entirely without any form of "agreement" other than how they are applied (see https://news.ycombinator.com/item?id=30560573). I wouldn't call distributed CRDTs "consensus."

Reply on news.ycombinator.com