Building a BitTorrent client from scratch in C#

Former admin of DC# here (forgive the sourceforge hosting-- it was a long time ago! https://sourceforge.net/projects/dc-sharp/). Great write-up! This was a fascinating read- thank you for putting it together.

One issue to be mindful of- the HttpWebRequest.BeginGetResponse method does not honor timeouts, and you are on your own to timeout the attempt. Consider using HttpClient, if available in Mono / .NET Core. Otherwise, see MSDN for how to do this:

"In the case of asynchronous requests, it is the responsibility of the client application to implement its own time-out mechanism. The following code example shows how to do it." See: https://msdn.microsoft.com/en-us/library/system.net.httpwebr...

I'm not sure if you have access to the ThreadPool class. In a bug that Microsoft's library had, I used the TPL Task construct to resolve this. See the pull request here: https://github.com/Microsoft/ProjectOxford-ClientSDK/pull/83...

Great work! Quite annoying actually. I finished my own implementation in Python at about 10pm last night, this would have been most useful. I'm no C# coder, but it's nicely readable, and this is a much better write up than I'm sure I could do.

If anyone who hasn't tried doing this before, the "official" BitTorrent spec docs, namely BEP-3 (http://bittorrent.org/beps/bep_0003.html), seem little more than a vague blog post turned in to a "spec". However, somewhat conversely, this has lead to is a wealth of articles describing how to do it.

The three guides I used were:

- A 2 part blog post which has a bit of a Python bent http://www.kristenwidman.com/blog/33/how-to-write-a-bittorre...

- The unofficial specs https://wiki.theory.org/BitTorrentSpecification, and

- An incomplete Python client https://github.com/JosephSalisbury/python-bittorrent

I didn't know of the RFC mentioned in the post, that would have also been really useful.

A lot of BitTorrent stuff for Python is remarkably hard to find in all the noise of Deluge, the original client, and libtorrent wrappers, but none that existed were sophisticated (or at least well documented) enough for my experiments, they have different focuses.

I never went as far as implementing my own BEncoder library, a billion seem to exist in multiple languages and install any BitTorrent Python library and it seems to come with their own copy. (I suspect due to the way BEncoder was bundled in the original client, see: https://pypi.python.org/pypi/bencode)

I also found a Rust implementation which seems not to compile, but is useful as I'm trying to teach myself Rust https://github.com/kenpratt/rusty_torrent I think the work to get it to compile might be minimal.

Very good! A few minor performance comments.

1. In your EncodeDictionary, you sort byte arrays by converting them to string. Correct but subeffective. See e.g. this: http://stackoverflow.com/q/19695629/126995 but add checks for nulls, authors of that code forgot about that.

2. You don’t need a dedicated thread to wake up every 1-10 seconds and do something small. Thread are expensive system resources, they own stack, cache misses are guaranteed then they wake up, etc. If your compiler supports async-await, use that instead + endless loop + Task.Delay inside the loop. If not, System.Timers.Timer class will do.

BEncoding and variants like REncoding are possibly one of my least favourite things ever. If you deal with the Deluge torrent client API you'll see it everywhere.

That aside, fantastic work on this, I think previously the only Bittorrent library for C# was an abandoned Mono project.

Here’s a similar write-up on building a BitTorrent client in Haskell: https://blog.chaps.io/2015/10/05/torrent-client-in-haskell-1...

If anyone is interested in a similar writeup for node.js check out my tutorial here: http://allenkim67.github.io/bittorrent/2016/05/04/how-to-mak...

Awesome write-up! I love C# and this was really well-written. Great work.

Pro tip: Use DateTimeOffset instead of DateTime. It's less frustratingly ambiguous than DateTime, and already has a Unix timestamp helper function if you're on the latest framework: https://msdn.microsoft.com/en-us/library/system.datetimeoffs...

Not sure if it's a part of the standard library, but the .Net variant of C# contains a sorted dictionary: https://msdn.microsoft.com/en-us/library/f7fta44c(v=vs.110)....

It's really nice to see a walkthrough of a non-trivial program all on one page like this. The clarity of the code and writing makes me want to port it to a different language because it seems like it would be easy with all the needed info in one place.

Heh, more than a decade ago I created a torrent file format library in .NET ... actually with VB.Net ... anyway, this is GPLv2 licensed. http://writtorrent.cvs.sourceforge.net/viewvc/writtorrent/wr...

It's really great work OP. I know this would have taken you a long time to do but part of me can't help but wonder if programming is becoming even more like paint by numbers than it already is.

For interested people there is a great write up on Tox protocol here https://toktok.github.io/spec

I wish there's a similar article in python.

I don't understand this encoding method. If say, a dictionary starts with d and ends with e, how do you know with "d3:key5:valuee" if the value is "value" or "valu"?

I've been considering trying my hand at creating a bittorrent client. This should prove to be most helpful!

Somewhat disappointing that it's just a console app. I'd love to be able to do cross platform C# desktop development. There shock be something equivalent to WinForms/WPF on OSs other than Windows.

Fun read, but using automatic properties might lead you down a path that isn't optimal;

Take this for example:

    public byte[] Infohash { get; private set; } = new byte[20];
    public string HexStringInfohash { get { return String.Join("", this.Infohash.Select(x => x.ToString("x2"))); } }
    public string UrlSafeStringInfohash { get { return Encoding.UTF8.GetString(WebUtility.UrlEncodeToBytes(this.Infohash, 0, 20)); } }

You have an automatic property and two 'properties' that actually perform work every time you call the getter (might be smarter to make functions of those, so you know it's not just retrieval of data, but work is done).

If you were to rewrite this a bit, you could make sure the 'work' is done only when needed, and the properties become actual simple data retrieval properties like:

    public class Hashes
    {
        byte[] _infohash;
        string _hexStringInfohash, _urlSafeStringInfohash;
        public byte[] Infohash
        {
            get { return _infohash; }
            private set
            {
                _infohash = value;
                _hexStringInfohash = String.Join("", this.Infohash.Select(x => x.ToString("x2")));
                _urlSafeStringInfohash = Encoding.UTF8.GetString(WebUtility.UrlEncodeToBytes(this.Infohash, 0, 20));
            }
        }
        public string HexStringInfohash { get { return _hexStringInfohash; } }
        public string UrlSafeStringInfohash { get { return _urlSafeStringInfohash; } }

        public Hashes()
        {
            Infohash = new byte[20];
        }
    }

Going further through the article, I spot many more items to improve; but let's not forget your did great work and the code is quite readable.

One thing that might help; is building some indexes to know how files are fragmented; you have the following code multiple times:

        if ((start < Files[i].Offset && end < Files[i].Offset) ||
            (start > Files[i].Offset + Files[i].Size && end > Files[i].Offset + Files[i].Size))
            continue;

If you'd build an index to know which piece hits which files, you don't have to enumerate this every time.

Another general remark is to always 'retrieve' an indexed item from the array and use that instead of keep calling the 'indexed' record.

So; do:

        var file = Files[i];
        if ((start < file.Offset && end < file.Offset) ||
            (start > file.Offset + file.Size && end > file.Offset + file.Size))
            continue;

The code becomes more readable and allows you to change the structure later on more easily since you don't have 100 references tot he same array now and only use an itermediate.

Is the site down?

If anyone's having issues, I've mirrored it at https://cheatdeath.github.io/research-bittorrent-doc/

edit: I'm the author, let me know if you have any questions.