Hacker News Reader

Hacker News

Comment by __turbobrew__ | Hacker News Reader

Full thread

__turbobrew__

Does anyone have recommendations on books/papers/articles which cover gossip protocols?

I have been more interested in learning about gossip protocols and how they are used, different tradeoffs, etc.

View on HN

Two interesting papers:

* Epidemic broadcast trees: https://asc.di.fct.unl.pt/~jleitao/pdf/srds07-leitao.pdf

* HyParView: https://asc.di.fct.unl.pt/~jleitao/pdf/dsn07-leitao.pdf

The iroh-gossip implementation is based on those: https://docs.rs/iroh-gossip/latest/iroh_gossip/

Thank you

https://thesecretlivesofdata.com/raft/

> https://thesecretlivesofdata.com/raft/

Are you suggesting to use raft as a gossip protocol? Run a replicated state machine with leader election, replicated logs and stable storage?

Raft is a consensus protocol, which is very different from a gossip protocol.

I'm sorry, I got confused.

I’ve worked with a few, and seen a few built. IME they are never really worth it. They make the system much more complex, fragile and hard to debug. The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale.

> The scale limits they aim to help are just not a big deal anymore. A small group of “coordinator” nodes to do this is much simpler and easy to build and scale.

What changed that made it easier?

Faster hardware (cpu and nic) and better low overhead networking APIs in Linux. One machine can manage really a ton of low volume connections.

10 years ago I've implemented SCAMP (a gossip protocol) in Clojure, you might find it interesting, the implementation is quite small https://github.com/cipherself/gossip

Small example from CZMQ - High-level C Binding for ZeroMQ: http://czmq.zeromq.org/manual:zgossip . Perhaps not self-contained, one might have to read the manual of other resources, such as zactor, zsock or of the application Zyre - Local Area Clustering for Peer-to-Peer Applications https://github.com/zeromq/zyre .

While not a book/paper/article, this is good implementation practice: https://fly.io/dist-sys/

Check out SWIM and its extensions: https://en.wikipedia.org/wiki/SWIM_Protocol https://arxiv.org/abs/1707.00788

Reply on news.ycombinator.com