Show HN: I wrote a BitTorrent Client from scratch | Hacker News Reader

Show HN: I wrote a BitTorrent Client from scratch

github.com

211 pointspiyushgupta5355 comments

I picked up programming in late 2023 and been enjoying it now. Wanted to challenge myself and set a stretch goal, so set out to build a bittorrent client.

Open articleView on HN

Great work! Just an FYI, you might want to limit the dynamic allocation size in the bencode decoder: since it's untrusted input (either from torrent or announce), a malicious input could DoS the client by requesting extremely large allocations during string parsing. A good upper bound could be the remaining length of the input, as a well formed torrent can't contain a string longer than the rest of the file.

thanks for pointing this out. I've added this in my to-dos.

You might look into (if you only care about reading it) writing the bencode decoder using Kaitai Struct [0] to avoid some of the common pitfalls.

[0]: kaitai.io

Excellent, looks clean and simple.

Suggestion: Add a simple usage one liner in the README on how to actually download a .torrent file with it.

  ./go-torrent My-Linux-Distro-Wink-ISO.torrent

Suggestion: Bonus points if you add torrent.ParseFromUrl

Everyone should do this for their own spiritual journey.

thanks for the suggestion, appreciate it.

Neat! There is this challenge on codecrafters that guides you through the process a little, provides tests and such, I've played around with it a bit during a free month they had, was fun:

https://app.codecrafters.io/courses/bittorrent/overview

This is cool! I’ve been thinking about something like this as well. How hard was it, and do you have a sense for how “complete” it is? Does it handle DHT and Magnet and all the crazy NAT traversal stuff?

I’m guessing the main obstacle for me has always been that I’m not sure what the complete list of features is to have a client that will just work for the majority of torrents in the wild. It seems like there are dozens of protocols associated with torrenting and I don’t even know what the full list is much less what each does.

it was challenging for sure. Took me almost a month to get acquainted with the protocol, how bencoding works etc, build a mental model and then eventually writing code.

Magnet and DHT are yet to be added.

In my experience, magnet is pretty straightforward. Dht is quite the rabbit hole, and it might be difficult finding clients that support dht (not everyone does)

How hard it is depends a lot on your experience, how comfortable you are with the programming language, and if you have an experimentation mindset. To offer a perspective, I also decided to build a Bittorrent client in Go last week, and I'm about 80% of the way of the posted project in a week. But then again I have a lot of experience with Go, extensive knowledge about protocols and networking, and experimentation has become a kind of second nature for me.

If you're interested, the main Bittorrent spec is actually quite small, you can easily read and understand it in an hour or so: https://www.bittorrent.org/beps/bep_0003.html

There's a lot of stuff "around" Bittorrent, like protocol extensions. But in my experience so far, many clients have different levels of support. E.g. they would try communicating using Protocol Encryption (obfuscation, actually) and if it doesn't work fall back to the regular protocol (if configured in the settings).

If you just want to download a Linux distribution (e.g. Debian) using the .torrent file on their website, it's relatively straight forward. It has just one file, there's a tracker, plenty of peers using the regular protocol, and you could get away not listening for connections when you're behind a NAT.

Naturally when you get into the grey area of content, then you'd need to parse magnet links, use DHT to discover peers, and perhaps you'd want to implement UPnP to try and automatically map a port on your modem. But that is also an iterative process, you can just build out one by one. The more features, the more peers you'll be able to discover and have successful exchanges with.

As a non go developer, may I ask why you're using older go version 1.21? Is there a reason to stay with older releases?

EDIT: It seems like it was deprecated 10 months ago

Windows 7 support is one reason to stick to older GoLang releases. A project in Go 1.21.4 or earlier will work on every Windows release and any computer made since 2009, whereas a version bump to v1.21.5 means it will only work on more recent computers and Win10 and 11 for no benefit.

https://github.com/golang/go/issues/64622

I think this is a reasonable take. Yes, people shouldn't be running Windows 7 as their daily driver. But if you can support it at basically no effort and without sacrifices that is the right thing to do. Supporting more platforms is a good thing, even if that platform is an old Windows version instead of an Amiga

> and without sacrifices

There are always trade-offs/sacrifices.

The Go team isn't making new versions just for fun. Each version since 1.21 has had improvements. Especially the fix/change to for loop variables in 1.22 is very nice to have, and helps preventing to write bugs.

If there's a reasonable expectation that users will use outdated platforms, it makes sense to support them. If there is no such expectation at all, why would one forego the improvements to the language and tooling.

There's always a price to pay.

In this case the 'improvements' aren't required for this particular project.

The README is likely AI-generated. The actual go.mod file lists 1.23.1 as the Go version[1], which implies a requirement of Go 1.23.1 or higher[2].

[1] https://github.com/piyushgupta53/go-torrent-client/blob/6130...

[2] https://go.dev/doc/modules/gomod-ref#go-notes

Go 1.21 includes an internal auto-updater that can compile later Go versions by looking in the go mod file itself.

So the README is correct.

Automatic toolchain switching does not trigger compilation of later Go toolchains; it only attempts fetching a blob (if available) and using that to perform builds[1]. If a system supports Go 1.21 but not 1.23.1 (e.g. Windows 7, mentioned in a sibling comment), then this project will fail to build on said system. Likewise, if a user on Go 1.21 has disabled automatic toolchain switching, the Go CLI refuses to build the project at all.

Overall, I would say the minimum required Go version is indeed the Go version declared in the go.mod, since that is the declared minimum required toolchain.

[1] https://go.dev/doc/toolchain#select

Very cool!

This brings me back to college. We did this as our final project for our networking class at Georgia Tech.

I've long lost the code for this, but the lessons learned have lived on :)

Projects like these are great ways to learn new languages too!

It's a great kind of project to try! I did something similar and I was humbled when looking at other people's bencode encoders/decoders. It's also a great way to check out language features! Of course it is quite futile to try to match all the features of large libraries, and I believe the compatibility between clients is already lacking...

Do you plan to release docs about the process? personally I'd be more interested in that than the code itself, as few people really go in-depth about the process, the approaches to implementing different aspects of the protocol, etc.

Did you do v2 and mutable torrents? Please, for the love of all that is good and wholesome, someone do mutable torrents.

Do you support magnet links?

Edit: ah, planned feature

not yet. I'll be adding soon.

How hard would it be to add a GUI to this? I don't think I've seen a lot of GO Gui implementations in the past

There's a bunch https://github.com/go-graphics/go-gui-projects

My personal favourite is an ImGui wrapper https://github.com/AllenDang/giu

The most featureful is probably unison, although I'm uncertain if anyone uses it outside of their own project (https://gurpscharactersheet.com), meaning documentation will be sparse https://github.com/richardwilkes/unison

Gio uses a different way of thinking about GUIs, used by Tailscale and gotraceui https://gioui.org

Wails is great if you're familiar with development on the web https://wails.io

The GTK4 bindings also look nice https://github.com/diamondburned/gotk4

Cogent Core also looks neat, but I didn't have the time to play with it before I switched over to using the Odin programming language instead of Go https://www.cogentcore.org/core

I personally had nothing but issues with Fyne (especially in regard to performance, across multiple computers and operating systems), but it's probably the most popular option https://fyne.io

I would like to echo performance (and other issues) with Fyne. Every time I try it again, I'm actually kind of baffled that it's so popular.

Last week I came across these Qt bindings: https://github.com/mappu/miqt

I have no personal experience with it yet, but am excited to try it out.

what did you refer for building this? did you read the protocol or did you go through other implementations? curious because it always makes me wonder how people build stuff like this from scratch

Interestingly (or maybe not) I started writing a Bittorrent client last week, also in Go. I don't really like AI/LLMs for coding, and my goal is to really learn from it. Not to just deliver some piece of code that an AI wrote...

So the first step is to figure out if there is an actual spec for the protocol. Well, in the case of Bittorrent there is a spec, but it's kinda... brief. Basically, this is the spec: https://www.bittorrent.org/beps/bep_0003.html

There are some additional extensions here: https://www.bittorrent.org/beps/bep_0000.html

It's very important to break the work up in tiny steps that each give some kind of a result. E.g. I started with parsing .torrent files, for which you have to implement bencoding.

That was interesting, because I downloaded the Arch Linux .torrent and found out it was basically incorrect: it has a url-list key which isn't mentioned in the spec. Then you have to go down a rabbit hole of some research trying to find out what the heck is up with that (turned out to be https://www.bittorrent.org/beps/bep_0019.html). Finally, was able to successfully parse the Debian Linux .torrent file.

Then came implementing the tracker Announce HTTP request and the peer protocol. The peer protocol was tricky, and that's when an experimentation mindset comes in handy. I stripped the announce URL from the Debian torrent and loaded it up in KTorrent, that way it wasn't able to find any peers. I used the "Add peer" option to add my client so I could see what messages it was sending me, and alongside I would implement sending those to KTorrent messages too. A lot of trial and error and debugging.

I have to confess that from time to time I had to ask ChatGPT about some small protocol details because they just didn't show up in the spec. There's also a lot of details that clients have implemented over time but which aren't really documented well. Mostly algorithms for which pieces to download, which peers to choose, how to "choke"/"unchoke", etc, is not well-defined. But web searches help out a lot.

Also an honorable mention to https://wiki.theory.org/Main_Page which has some nice information.

I'm now at a point where I can fully download and upload a torrent with my local KTorrent, so the next steps are to actually get peers from the tracker, and some algorithm to download pieces and store them in a file.

Hope this gives some insights. Let me know if you have specific questions about the process.

Could this be used as a library?

I am not the one who created the post, but currently it doesn't seem to be usable as a library because most of the code is in a directory called "internal", which isn't accessible when using the repository as a dependency in another project.

Stub for offtopicness

sounds very cool. Good luck!

[flagged]

Or like a go beginner, which is fine

Scanning around their other repositories the persons been programming for a few years now. There are ‘.cursor/rules’ directories in some recent repos.

I think it’s a reasonable hypothesis that “I wrote a BitTorrent client from scratch” may be “I produced a BitTorrent client from cursor”.

Yes, very strange. There's no problem with using AI to build your first app and leaving the generated comments in the code is fine. But the number of comments on this thread that begin "This is so cool" is very suspicious.

It doesn't seem wild to me that people would post "This is cool" on a ShowHN post. I did [here](https://news.ycombinator.com/item?id=44265915), and as far as I'm aware, I'm not an AI.

Would you like more information about how to identify AI comments? (kidding)

"convert length string into an integer" is a machine generated comment?

I've been writing code for 15+ years, this made me laugh my ass off. Comments are great, I don't read comments but I write them for others, especially for open source code. Atoi may be something you and I and a whole bunch of others know but people who don't it's a fine comment. Relax! :)

In the tests it more obvious:

You can see here for example: https://github.com/piyushgupta53/go-torrent-client/commit/61...

and some strings coming from crawled resources like: lengthi12345e4 but slightly different tokens (like 25 instead of 35 etc).

Gemini Pro 2.5 even gave me the prompt:

> If you asked me, "Generate Go unit tests for a Bencode decoder function called Decode that takes an io.Reader and returns an interface{} and an error. Cover strings, integers, lists, and dictionaries, including common error cases and nested structures" the output I would strive to produce would look very much like the code you've shown.

> It's a good example of well-written Go tests, and that's the kind of pattern I've learned to recognize and replicate.

and a lot actually matches when you ask from a fresh conversation.

So most likely Cursor + Gemini 2.5 Pro, but I cannot blame, I spend 100% of my time with Claude, and I take ownership of the code.

"TODO: We'll develop the actual functionality as we develop each component" lool

It's hard to say honestly. I don't call any project AI as it's just too hard to tell. I write lots of comments in my code too so it's hard to call anything AI without a person stating they used it.

Claude is decent for sure, but I always say with AI, learn the math before jumping to a calculator.

That comment is a strong sign that this was AI-generated. LLMs have the tendency to leave superfluous comments even when the code is self-explanatory. In this case, strconv is a well-known stdlib package, and anyone reading this in their IDE would get the documentation for what it does. In fact, all of the comments in this function and in most of the file are redundant, and I would point this out in a code review.

But, of course, this was vibe coded, so it's unlikely a human actually reviewed it.

Clean code! Very nice :)

what's up with the amount of new accounts praising this project?

Seems like someone (OP or not) is testing how good they can use HN for free advertisement.

odd indeed

I only see two green usernames. Have others been deleted already?

No seeding, no DHT, no magnet links, no uTP, no extensions. At this stage it is BitTorrent downloader, not a client.

Using P2P networks in download-only mode, so called leeching or free-riding, is discouraged.

Reply on news.ycombinator.com