Hacker News Reader

Comment by lqet | Hacker News Reader

Very cool, and surprisingly simple. I was always mystified as a kid how these file-sharing protocols actually work, but if you just look at a torrent as a collection of fixed-size pieces you are asking peers managed by a tracker for, it becomes really straightforward to implement.

Now I wonder how clients protect themselves against abusers (i.e.. people who never upload a single bit but only download). I often noticed that when I set the maximum upstream to 0, clients would stop sending me pieces. Do clients share a predefined list of configuration parameters with each other or with the tracker (Max upstream, max downstream, etc.)? Or is it something more sophisticated?

View on HN

Sufficiently evolved torrent clients follow a tit-for-tat protocol:

- If you're nice with me I'll be nice with you

- If you're mean with me I'll be mean with you

This is the best answer for the Prisoner's dilemma (https://en.wikipedia.org/wiki/Prisoner's_dilemma) on the long run, ie a situation where peers don't trust each other but will both gain if both cooperate.

In bittorrent it's typically implemented as follows: peers start by sending a very minimal content, and see what the other replies. If they reply with low enough latency then slowly increase the amount of content that is sent and see if they reply with the same increased amount of content; if they do, continue up to the max of what the network link allows (in combination with other peers of course). If at some point the other peer doesn't send something equivalent (even though we know they have it and we asked for it) then that peer can be cut off for not being cooperative.

Situation is different for seeds of course, because they have everything and want nothing, but they can have a similar behaviour -- start sending a little, increase slowly over time

Did knew, that the client does that with the tit-for-that.

In an ideal world the network would prioritize to upload to high speed like webseeds, so the webseeds can distribute even faster for everyone else.

Or do I make a mistake in my mind and the network already distributes efficiently and it is not needed.

A single node distributing everything "because it's fast" isn't really the architecture that bittorrent is aiming for, and I don't think it's one we should be aiming for.

Try downloading any well distributed torrent and you'll see your bandwidth capacity automatically maxxed out. The gradual decision with an emerging selection of better sources always leads to that. Bittorrent doesn't really have an efficiency problem

> [tit for tat] is the best answer for the Prisoner's dilemma on the long run

This is fascinating. Have there been any simulations about this? I'm sure they've looked into this in game theory, but I'm wondering if you have a big swarm of torrent clients, does the scale of the population change the outcome, or is it the same?

(It seems like it would be wise when designing the torrent clients to run such simulations, so I'm thinking this probably has been done at some point.)

Check this out for a nice explanation of why tit for tat is a good strategy for the repeated Prisoner's dilemma: https://ncase.me/trust/

Thank you, this is great!

It's only the best answer if the other party is bothering to seek the best answer in real time, and in ways that you can't predict. If your opponent is governed by some algorithm, which is the case for torrent clients, you may need to depart from tit for tat in order to break them out of a needlessly adversarial mode. Or, if you're feeling like the bad guy today, you might need to depart from tit for tat to put them in an irrationally cooperative mode.

If a peer doesn't respond quickly enough they can be disconnected from and marked as bad. Even if they had good intentions their latency might be too high and it's good to disconnect from them.

Check out this very nice link from a sibling comment: https://ncase.me/trust/. You'll see that an adversarial node either is too adversarial and will be cut off, or still gives just enough to not be cut off but if there are other, more cooperative nodes, they will be favored. Being adversarial doesn't work in the long run.

> Now I wonder how clients protect themselves against abusers (i.e. people who never upload a single bit but only download).

It's called leeching, and it depends. It's typically considered good etiquette to upload as much as you download and that's usually enough, but it can be enforced

https://en.wikipedia.org/wiki/Leecher_(computing)

I have the opposite problem, many things I download seem so well-seeded (seedboxes?) that I rarely achieve any sort of respectable ratio even if my uplink idles most of the time. It might be client problem but I haven't found much anyone discussing this. TAKE MY BITS!

Over here (austria), you are pretty free to leech as much as you want, but as soon as you actually start seeding at any capacity, people start paying very close attention.

See my other comment, good seeders carry most of the network these days. Upload bandwidth has become huge since BitTorrent first debuted so true P2P for resource reasons is not really a necessity anymore.

I've often considered writing the equivalent of a robotrader for this. If you're paying for a sedbox and want to use it to the fullest re: maximizing your ratio then some automated attempt to predict demand and start seeding the right thing early would likely go a long way.

Unlike predicting the stock market, competition wouldn't be very fierce, and I think the tracker API would give you most of what you need.

It's in the long list of things I'll do if I ever find a big pile of time.

If the torrent is already well-seeded it's ok, a 1567th seed won't make a difference so don't bother with them. It's the less seeded torrents where you can put some efforts and do something good.

IIRC, when both peers only have some subset of pieces, they are meant to “trade” with each other, and so will eventually drop low value peers for higher value ones. Aside from the adversarial case, this helps optimise for being nearer to the people you are sharing with, since torrents are global.

In practice, BitTorrent really needs seeders who have downloaded the entirety of the file to be fast for everything except really popular downloads, Seeders don’t really check for fairness and will typically upload to whoever they can the fastest (with a limit on number of peers).

There’s an even more adversarial case because the unit of validation (a piece with a fixed hash from the spec) might be bigger than the chunks that are being shared individually. So it’s possible for a peer to fake having pieces and upload garbage data instead, and they wouldn’t be caught since different chunks came from different peers.

For the regular case, 5-10 seeders will hold up an entire torrent indefinitely and without any issues. The contribution leechers make drops off very quickly if good seeders exist.

My algorithm did favour the best peers (both upload and download) but a few years ago switched to a "seeded" ordering to prevent bad behaviour clients dumping or starving new peers.

The garbage uploaders are not an issue. You can isolate peers to pieces and remove them with certainty, or use a technique called smartban which uncovers bad peers very quickly.

Sure, I didn’t mean to suggest that it was impossible to handle, just that it was an edge case that you needed to explicitly handle and which a minimal implementation of the protocol might be likely to leave out.

Wouldn't the peer uploading garbage data be caught immediately because the garbage wouldn't hash to the correct value? I imagine most clients would stop talking to such a peer rather quickly.

Torrents are split into pieces, each of which has a hash that allows it to be validated independently.

However, these pieces can themselves be large, so it is often recommended to split them into smaller chunks that are sent one at a time. You cannot validate a chunk independently, and if you assemble a piece from chunks from multiple origins, you can’t immediately tell which origin is bad.

You need to explicitly handle this case, like the sibling comment mentioned by isolating peers who may have sent bad data and forcing validation.

There's private tracker which forces their user to upload.

Reply on news.ycombinator.com