Building a BitTorrent client from the ground up in Go (2020)

I've recently started learning Go and chose a project-based approach. I've been reading "Writing an Interpreter in Go" and "Powerful Command-Line Applications in Go" and it's been loads of fun.

One of the things that made the bootstrapping click for me was realizing there are in fact servers that DHT clients talk to once to find an entrance to the network. They all have a list (such as https://github.com/qbittorrent/qBittorrent/blob/c80238d66ff3... )

One of my favorite things about Go's language design is that difference between really good code and average-level code is much smaller.

They're big fans of bittorrent, it appears.

The l looks taller than the 4 to me, which makes it clear it's not another number

I had the same confusion! In my browser, with the default monospace font, the two characters are so similarly as to be indistinguishable.

Reading the next example, showing an encoded dictionary, was what made the penny drop. Bencode doesn't go out of its way to be visually accessible, so different font choices would help.

Previously: https://news.ycombinator.com/item?id=21958359

You can normally control the number of peers you share with, so I guess if 50 peers are connected but you only want to share with 10, then when one completes you'd send an unchoke message to another peer to tell it you'll share with them. Meanwhile the other 39 will stay choked. I assume that's what's going on here.

This was posted recently: https://github.com/codecrafters-io/build-your-own-x

You may be thinking of private trackers, where "ratio" is the currency of participation in that community. Debian, Libre Office, and even Humble Bundle care more about the community of seeders sharing the bandwidth load than whether anyone seeds their ratio

Was that worth a downvote? At least, if my question was that stupid, could you point me to the part of the blog post where it shows how pieces are sent to other peers?

Hi I'm the original author. Sorry to see that you got downvoted because it's a good question and a real limitation of my project. I might get around to building a follow-up some day with upload/sharing since that's where the actual interesting stuff happens (and not just shuffling bits around, with a shitty download algorithm tacked on like in my post). No real plans though. If I do that then I might have to rewrite some of the code and edit parts of the blog post, since I've grown as an engineer and writer since I made this two years ago.

As for your question of "Is that tolerated?" the question is a social one. As the sibling commenter pointed out, if you use a private tracker, you're part of a community that often has a social contract (implicit or explicit) that participation means you need to contribute back and have a good seeding ratio. In that case, only downloading -- like my client does -- is not tolerated and might get you banned. That's why I chose to demo with torrenting a Linux ISO, since those tend to be public, have a lot of seeders anyway, and don't really care how much you upload or download.

There's nothing in the protocol that says you have to do it, and indeed, there are even "selfish" algorithms that try to maximize download speed while uploading nothing in return: https://www.wired.com/2007/01/bittorrent-bullies-bittyrant-a...

Even if the peer isn't behind a NAT (or on IPv6), you still need to pass the firewall, which won't let incoming packets through. This can be solved by simply connecting from both ends simultaneously, (first attempt fails but opens the fw, second attempt in the other direction succeeds)

You only need hole punching if both peers are behind a NAT (TCP hole punching is substantially more complex and fragile than UDP). You can reduce the need for hole punching if you complicate the signaling protocol (eg "please dial me at <addr>").

That said, bittorrent is quite resilient to NATs in practice, because since it's many-to-many, chances are even NATed clients will find many non-NATed clients.