Truly decentralized bittorrent | Hacker News Reader

Truly decentralized bittorrent

torrentfreak.com

114 pointsRhapso19 comments

One of the by-products of centralized trackers is allowing private trackers to exist. There are two reasons for them

  * Promoting seeding
  * Basic invite-only protection from IP gathering oponents

Decentralized alternatives don't provide that. Think about the Wikileaks "insurance" torrent, there are plenty of government agencies sniffing. Even with peer blockers they can gather client IPs with the tracker, through DHT or peer exchange.

It's very interesting how this will shape the future of Internet and the Web.

OneSwarm has an interesting approach to that problem, using short hop onion routing when sharing data with unknown nodes.

Tribler promotes seeding by gossiping upload/download totals between peers and is experimenting with various methods of calculating reputation based on these statistics.

Haven't all clients supported DHT for a while now? It's good to see a client take the extra step and decentralise search as well, though. Hopefully this won't go the way of KaZaA/Limewire/Gnutella, but even if it does, we'll still have private trackers.

Its not using a DHT, its a gossip based network. Its simpler and more resilient to random faults but uses more space and bandwidth. As far as I know there hasn't been much research into their resilience against systematic or malicious faults.

Does anybody have a more technical description of this? as they say, the devil is in the detail

From my comment on the dupe: http://news.ycombinator.com/item?id=1985431

Most of the underlying protocols are described here, though it is a bit out of date:

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.78....

Essentially the core protocol is a gossip overlay. This generates random connections between peers in the overlay. Over these connections peers transmit information like recently seen torrents, recommendation lists, download/upload totals for other peers etc. Each peer proactively distributes known torrent files so that every peer builds up a database of torrents. Then search just works by broadcasting queries to nearby peers.

The algorithms used have dozens of applications outside of filesharing. I wrote my MSc dissertation on a similar subject:

http://scattered-thoughts.net/one/1283/644001/538941

The code for Tribler is naturally very specialised to their purpose. At some point I want to sit down and reimplement them in such a way that they can easily be reused and recombined for other applications. I have some vague ideas about building a prototype trust-based distributed database for dot-p2p.org based on similar principles.

Sounds like a real Skynet.

Very similar, except that Skynet was a vast, distributed, self-aware intelligence controlling most of the worlds machines and Tribler is a file-sharing network. But they both run on computers ...

> Essentially the core protocol is a gossip overlay.

Is this a broadly similar idea to NNTP 'flood fill', where once you have a peer (discovered in some way), you exchange articles (or in this case torrents) you each have - so they propogate to all connected nodes in the graph?

If so, doesn't this have problems scaling? (All nodes receive all torrents)?

> All nodes receive all torrents?

Each node has a subset of all torrents. You flood searches as well. There is also a second overlay on top of the first which groups peers by their similarity in taste, with the aim of reducing the number of hops necessary to find what you're looking for. The taste similarity is also used to actively propagate recommendations so that stuff you're interested in is more likely to move towards your immediate vicinity.

I would recommend scanning through the paper to get a more accurate idea of how it works. My knowledge of Tribler is pretty out of date.

wasn't the biggest issue with the first gnutella networks that search through flooding basically does not work? Is this somewhat accomodated by the fact that now we, well, can basically keep much larger databases?

Bigger databases, proactive distribution of torrents and similarity based clustering of users. Tribler is too small as yet to know if it will work but it certainly looks like an improvement on gnutella.

One big question: did it work for you? Did you download/watch something?

I've used it a couple of times. Their streaming video is pretty useless most of the time but downloading stuff and watching it offline works fine. I can't tell how well the search works - there are only a few thousand active users so its likely that the stuff I'm looking for just isn't there. It might be worthwhile for them to also run searches on tpb etc and then import the torrents, otherwise they are going to suffer from network effects.

So, no practical difference from any regular client at the moment.

How is it different from DC++?

The core principle is similar. DC++ has centralized hubs which handle searches and upload files directly to users. Tribler has a distributed search engine which runs on every peer and uses bittorrent to share files. Effectively every single Tribler node behaves like a DC++ hub. As a result it is much harder to take down.

A minor nitpick is that DC++ is the open source client. The protocol/network is called Direct Connect.

http://en.wikipedia.org/wiki/Direct_Connect_network

That sounds a lot like Gnutella and it's worst own enemy: being so overloaded with search traffic that the rest suffers.

Reply on news.ycombinator.com