Libp2p – A modular network stack | Hacker News Reader

Libp2p – A modular network stack

libp2p.io

201 pointsespeed33 comments

What are some concrete examples for which this would be useful?

Near the end it says it's being developed by people in the IPFS community. Does anyone know for what they plan on using it?

Hi, libp2p dev here!

Yes, we use it today! libp2p is the P2P Networking Stack of IPFS that we refactored into its own project so that others could take advantage and be part of its ecosystem as well.

It is what gives IPFS and other projects the P2P connectivity, support for multiple platforms (including Browsers) and more! There are 3 implementations interoperable today at different stages of completion (Go, JS for Node.js & Browser and Rust). Full overview here https://www.youtube.com/watch?v=CRe_oDtfRLw

Also, Chris Pacia recently delivered an amazing talk at QCON about libp2p https://www.infoq.com/presentations/blockchain-libp2p, I totally recommend checking it out.

You can find more talks of examples and use cases at https://libp2p.io/media.

Give it a spin and post feedback on github.com/libp2p. We are also on #libp2p on freenode, come say hi!

The only thing I'm missing is a C++ implementation of libp2p.

Ethereum is the only example I've found that comes close, but it's poorly commented and only implements a small set of details of libp2p.

I use libp2p extensively for an upcoming open source project, Melodyne[1]. While there are some semantics that are annoying to deal with in the Go bindings, it is the most feature-rich p2p library I have discovered to date.

1: https://github.com/joshumax/Melograph/

It is what IPFS is built with, essentially they built out IPFS and made everything modular. It is being used in a lot of blockchain projects since it is so easy to develop with and write your own protocols.

Developing with the library essentially comes down to protobuf serialization logic inside a router with the path being the name of your protocol, just like you would in rails or node, everything else is handled by the library. To talk to a peer over your protocol you just need their address on the network (public key), your protocol name and to serialize your protobuf data to be written over the network, the libraries handle dialing/routing through connected peers if its not possible to directly connect to the peer.

If you find this cool you might also like Ouroboros[0]

[0]: https://ouroboros.ilabt.imec.be/index.html

Last time I looked at this, the code in general and code you needed to use it was overly complicated for what it does.

So what about this time?

I went to look at their Go implementation but the libraries are each split into separate Github repos (super annoying) and some of those appear to be non-public/404. Very strange.

> (super annoying)

Also found it very annoying using it. Coupled with the multi-whatever repos, that they make backwards incompatible changes, and they use IPFS paths in some of their imports (iirc) made for a non-welcoming library experience.

> (super annoying)

Why would anyone store different laguage implementations in the same Git repo?

Repo should be organized around project, not language/frameworks. It’s perfectly normal to have library +Python client + Go client in same repo.

Most of the time, I have a repo be an individual unit of testing. I.e. if the python client is broken, does that mean that the go client is broken as well? A large part of this is because its much easier to support testing multi-repos than mono-repos. Multi repos are generally easier to test because you run the entire suite of tests on every pull request whereas in a mono-repo you have to figure out which tests need to be run. All this isnt to say that its impossible (or not beneficial), but that its easier to have multiple repos with the current testing infrastructure (specifically looking at Jenkins). Another difficulty with mono-repos is merging code when something is broken, how do you know whether you should merge or wait for the bug to be fixed?

Is it? The Google repo posted by 'orangea is maybe the first one I've seen.

I mean, let's flip it around: when I need a lib for language X, why would I want to download it also for languages V, Y, and Z (+ all language-specific cruft)? Especially in this day and age, when dependency management in project involves build systems pulling repos automatically?

Or, put another way, library is one project. Python wrapper/client is a different project. Go wrapper/client is yet another project. Etc.

I can't think of any library for which this is the case, could you give some examples?

LLVM has python, go, ocaml and c bindings all in the same repository https://github.com/llvm-mirror/llvm/tree/master/bindings (the c bindings live somewhere else)

https://github.com/google/flatbuffers

In this case, the project (at least from the Go standpoint) is broken into multiple sub-projects, each within their own repo. In Go Land, you would more typically see each component library put in its own subdirectories off of the main repo. The main repo could contain libraries for multiple languages or just one, but the subcomponents would be organized as subdirectories, not separate repos.

Any idea why this is upvoted to make the front page this time?

I'm glad it was. I hadn't heard of it before, and now I'm watching the introduction video to learn more. I'm not really familiar with IPFS, but what I do know is that it has had some consistent buzz for a while now.

no C or C++? Looks promising but would love a C++ version

Same here, except I prefer C since that makes it more accessible.

That would also allow others to write clients for their favorite languages.

I'd also prefer a C version, but I think it's possible to call a Go function from C [0]. I don't know how well that would work though, since Go requires a runtime.

[0] https://github.com/golang/go/wiki/cgo#calling-go-functions-f...

There is also a Rust implementation of libp2p[0]. Exposing C functions from that library is probably the most "proper" way currently feasible to use libp2p from C.

[0]https://github.com/libp2p/rust-libp2p

C also requires a runtime, it just happens to be a very tiny one.

Simple, C++ with an additional extern "C"{} set of entry points.

C++'s improved safety over C, while providing such accessibility.

We are after all talking about network protocols, where security should be number one feature.

Would be cool if we had an ESP8266 and ESP32 friendly implementation for IoT

I've tried to use the Go version of this a few times and the custom package manager always mucks it up.

...which networking library is non-modular?

Just heads up, people should know libp2p (unless something has changed?) still has flooding problems.

There are other P2P transport alternatives, check out SSB's (it is really good!) or ours (gun.eco/docs/DAM) which prevents flooding. WebRTC and websocket implementations exist.

Hey Mark, pretty sure you're talking about floodsub, which is a naive implementation of pubsub of top of libp2p, not actually 'libp2p'. Libp2p is quite a bit more than a simple p2p pubsub library.

In any case, floodsub is just one option you can use for pubsub on top of libp2p (one with a notably high level of reliability and minimal cross network latency for infrequent messaging). We also have a tree pruning gossiping implementation called gossipsub. Take a look at the spec: https://github.com/libp2p/specs/blob/master/pubsub/gossipsub...

That is very interesting spec, but also alarming. It contains:

> The meshsub router offers a baseline construction with good amplification control properties, which we augment with gossip about message flow.

This is notoriously hard to get right. How to prevent spam, posioning or Sybils in your gossip network? Scientific work on p2p overlays and gossip is hard to publish now at venues like Usenix or NSDI. After Newscast from 2003 it seems you need strong identity to make these systems attack resilient. An unsolved problem.

Thank you for this insight! We will put a university student team on exploring this likely vulnerability in IPFS/libp2p.

Nice, thanks for clarifying, I stand corrected. This is great to hear/see, do you guys also have benchmarks yet?

Reply on news.ycombinator.com