Hacker News Reader

Comment by AndrewDucker | Hacker News Reader

I do wonder why a more decentralised approach hasn't taken off.

> I do wonder why a more decentralised approach hasn't taken off.

In a way, it has!

Compared to what it was when it launched 10 years ago, or even what its predecessors (e.g. Napster, Limewire) were, the current TPB is almost completely decentralized, with only one exception.

It stopped running its own tracker years ago, relying on other trackers that don't allow for searching (thus absolving themselves of some of the legal risks). And TPB has used DHT[0] for many torrents for a while, theoretically eliminating the need for trackers altogether.

So in 2014, The Pirate Bay is now effectively just a search board for magnet links, with usernames and comments attached only to provide some modicum of reputation (and therefore quality). Those magnet links point to content which is then exchanged either in a truly distributed fashion (DHT) or through these third-party trackers.

So the centralization doesn't provide "core functionality" to torrenting per se - it just provides a degree of quality. Nothing's preventing you from sharing magnet links on other forums (heck, you could paste one in an HN comment). But someone needs to be able to trust that what you're posting is, in fact, the content that you say it is and not a virus, honeypot, etc. Any forum that a group of people trust could serve this functionality (Reddit, HN, Yahoo Answers, etc.[1]) instead of TPB. The competitive advantage of TPB is not the content that it has, but the reputation that it's built over the last decade.

There are Silk Road-based alternatives, which use the Silk Road model for guaranteeing reputation. I guess Tor isn't technically "decentralized", but it'd be a small step to modify the Silk Road approach to broadcasting the actual magnet links themselves.

If you want to decentralize the last remaining centralized piece of TPB, you'd have to solve the problem of decentralizing reputation. This is something that's been worked on in a number of areas (e.g. Bitcoin) but remains unsolved. Perhaps the next TPB will use the Bitcoin blockchain, and this could somehow "solve" the issue, but there are a number of kinks that would have to be worked out.

[0] https://en.wikipedia.org/wiki/Distributed_hash_table

[1] Notice that these are all centralized

Reputation? For my use case, I evaluate a torrent based on the number of seeders.

You mentioned that anybody can post a magnet link on any forum. Yes. But where do they get that magnet link from in the first place? I mean, obviously the person who creates the torrent has it. Is that the only way?

You mention DHT. My torrent client does that. Hm, is there a client that I can use to "search" DHT? Failing that, is there a way to dump a list of what hashes/magnet links/torrents my own client is currently hearing about?

Sorry for not just googling this, but there's an awful lot of cargo cult information about torrents out there...

You could create a DHT with all known torrents. Kademila was built basically with that in mind. The Kad network works pretty well, but it does face some problems with spam; just sorting by seeders doesn't cut it since a spammer can fake as many seeders as he'd like.

http://www.tribler.org/

DHT is a way to discover peers who also have pieces of the same torrent you are downloading. It's an alternative to using a centralized Tracker to tell your client about who has the pieces of the file you need.

Yeah. When someone tells me "we must stop this sharing of valuable content!" I answer with something like "Here is all anyone needs to download your content: magnet:?xt=urn:btih:e6da54b1ad507ec6217610dbc71c248a1a49f925"... or whatever the magnet link is for the current LibreOffice ;-)

Basically, centralization is needed for deleting fakes, viruses, wrongly labeled stuff.

Totally decentralized approach will have all the negatives KaZaA had in the 00s ("matrix.avi.exe").

Decentralized with quality control would be a hard but interesting problem. You could do upvotes/downvotes using cryptographic signatures use a little web-based authority to let people know whose signatures are reputable.

Maybe a vote weighting that gradually gets heavier, the more honest your account is measured - you are observed upvoting good content, and flagging spam or viruses (imagine if the top HN commenters votes counted more than regular commenters...)? And rapid banning for those who upvote spam. Maybe you unlock additional feaures (like HNs downvote ability as you level up with karma). It might create a "race to the top" if you published a "karma list" of the top accounts... sort of like HN has with the leaders list https://news.ycombinator.com/leaders

The problem with those is always going to be sybil attacks: anyone can create tons of accounts and upvote themselves. Even cross-verification can be defeated: you just make a small network that upvotes each other in some obfuscated manner.

I think there are only two long term solutions:

- Introducing some kind of proof of work -- e.g. you do work to downvote/upvote;

- Some very "localized" reputation -- e.g. you trust the friends of your friends more.

Those ideas are behind the Bitcoin protocol and the Web of Trust, respectively.

In this case each would have it's problems: proof of work is inefficient by design and needs a good hash function not to be exploited; local reputation by design makes it hard to find new/unrelated content.

I don't think "proof of work" works here because upvoting/downvoting content needs to be cheap... how do you make it expensive for a dedicated malicious force but cheap for legitimate ratings? Proof-of-work as a sort of "sign-on" to enable propagating a new "account"? That is, you need to have 1 Ghz-hour of wasted computation in order to introduce a new cryptographic hash into the network as a sort of "payment to create an account"? Again, a dedicated spammer could have a machine farming accounts 24/7 while this "payment" to create an account would be frustratingly expensive to a new user.

The user wouldn't have to pay anything to create an account, just to vote. Sure, spammers can farm large amounts of reputation through botnets, but that's really expensive: you're putting a cost on it. The returns are already not so high for disseminating this kind of spam.

Having no reputation wouldn't mean you can't do anything -- just that what you do is less trusted and has lower priority.

By whom? Remember, we're talking decentralized. In Gnutella and the like there is no authority.

So if you want to take the Pirate Bay approach of providing a small authoritative list of "trusted moderators" through the web and have all the users use that filter, you have to keep it simple so that it can be easily replicated if the web-component gets taken down.

Complicating the web/authority component means mirroring it and resurrecting it becomes hard.

I've been thinking about this problem, too. (Reputation in a decentralised & anonymous web.) What would stop someone from creating fake users? They could easily form a realistic 'social network' over time, giving them some legitimacy. Together a large number of fake users can defeat any form of voting / quality control.

I find this stuff very interesting, let me know if I'm missing an obvious approach.

Well, obviously PGP has the whole "web of trust" thing but that's a bit hard to use. You can use crypto to "sign" things, so simply "signing" your upvotes and downvotes attaches a person to a vote. Then you let users upvote and downvote other users building a web of trust and a black-list. Signed-upvotes and downvotes would get passed around in the swarm itself as content.

The hard part is the new-user bootstrap. They need a starting list of trusted people - as they pull down information from the swarm, they get a good picture of the "web of trust". Then they can look for signed upvotes or downvotes on any content they're researching - that is, instead of just looking for the latest Game of Thrones vid, the system also looks for all upvotes and downvotes about each particular copy of the latest Game of Thrones vid and compares it against the user's personal "web of trust" to know which votes are respectable. The user sees this as just "here's the most popular copy".

Of course, passing around millions of upvotes and downvotes P2P would not be an easy problem.

Thanks for the reply. This sounds a little like the page-rank algorithm. :)

What stops someone from creating fake users on TPB, though?

It's not entirely implausible that authorities might attack the certificate provider. It's a single point of failure... the MPAA decides to fuck with it, gets them shut down, and suddenly no one can filter comments anymore?

Point, but that's a much smaller system to mirror. It can be static. You could bundle a cert list into the client, even.

a crowd funded approach could work to filter out "bad content".

Same way as here on HN, people could vote (plus or minus) to help rank the content.

You'd have to mitigate against the Sybil attack. Crowd funding is one way (ironic since I think you meant crowdsourced?) but as we know from SEO optimisation, all that does is force you to pay to get your spam/malware promoted.

What do you mean was the worse negative: downloading an exe, or downloading The Matrix? ;)

Thats a board statement.

TPB offered a means of verifying uploaders so that people could trust content. The current P2P system is distributed, however the trust isn't. Magnet links need to add a digital signature of the uploader so people can have faith in them.

However having a digital key means there is a private key that can tie an uploader to lots of illegal activity. Uploaders might not want that.

Reply on news.ycombinator.com