Hacker News Reader

Comment by nodja | Hacker News Reader

I assume this is being posted because qBittorrent recently released version 4.4 which includes libtorrent 2.x and therefore support for BitTorrent v2.

Hopefully we'll start seeing hybrid torrents/magnets in trackers soon. Although these are mostly automated so I'm not sure how fast the tools will catchup.

One thing I'd like qBittorrent and other torrent clients to do is to allow users to upgrade v1 torrents to hybrid torrents if they're fully seeded, this would allow cross polination for old torrents where peers are scattered between the original torrent and batch torrents, or where trackers add extra files causing the v1 hash to change.

View on HN

A bunch of open-source projects would like to use v2 torrents, but because Transmission is so popular and has zero support for v2, nobody can use them. Transmission's dev team hasn't said a word on the matter, ever, in four years:

https://github.com/transmission/transmission/issues/458

Elsewhere someone quipped:

> That transmission bug is 4 years old and they've never announced plans to prioritize it (granted the effort is surprisingly high; they've never allowed empty dict keys in bencoding). I think either they either don't know or don't care that their market share makes them an adoption blocker.

In the beginning I used µTorrent, until it got fucked by whatever company that bought it. It was easily the fastest and most resource-smart client I've ever used. After that I tried a bunch, Deluge, Transmission, Azure and plenty more, but only qBittorrent seems to be a suitable replacement (and is open source as well). Now it has BitTorrent V2 support too :) You should really give it a try.

Ironically, the company that bought µTorrent was BitTorrent, Inc[1]. They're the geniuses also responsible for forking IPFS, renaming it to "BTFS", and not acknowledging IPFS as an upstream in any way[2].

1. https://en.wikipedia.org/wiki/Rainberry,_Inc.

2. https://twitter.com/juanbenet/status/1250634833258143744

BTFS is the product of an entirely different group of people than the ones who were responsible for the uTorrent purchase. BitTorrent Inc was itself bought by cryptocurrency company TRON in 2018. Few people who worked at BT Inc prior to the acquisition are still there, and AFAIK none of them ever worked on BTFS.

Different team of people, all working in the same company, correct?

Technically yes, but as is typical with an acquisition there's residual divide between projects which originated from the BT Inc side of the company and those from the TRON side. BTFS is very much a product of the TRON side despite using the BitTorrent branding.

Wow, that's a bummer, had no idea the people behind the hollowing out and filling with shit of µTorrent include the actual creator of the protocol itself.

If it makes you feel any better about it, that tweet thread is from 2020, and the actual creator of the protocol itself hasn't worked there in a day to day capacity since 2017.[0]

[0]https://en.wikipedia.org/wiki/Bram_Cohen

It makes me not worry about this BTFS thing, but utorrent going bad was around 2010.

qBT also has a nice WebUI that has feature parity with the desktop version (or at least near enough that I don't miss anything), which is very useful. Other clients (e.g. Transmission) do too, but I like qBT the most here.

I agree, I put it on an rpi and it works great from any computer/phone in the house that way :)

I just use the version of µTorrent before the company was sold.

Some private trackers require updated software in order to connect to them. And also, you're vulnerable to bunch of security issues if you connect to the public internet with a very old µTorrent version. Do you really run a 10+ year old version of µTorrent without any drawbacks?

> Do you really run a 10+ year old version of µTorrent without any drawbacks?

Absolutely.

> I think either they either don't know or don't care that their market share makes them an adoption blocker.

The truth seems to be that between 2014 and mid-2021 the "transmission dev team" was a single (probably part-time) maintainer:

https://github.com/transmission/transmission/graphs/contribu...

Wow I didn't realize transmission had its own implementation of the torrent protocol, I thought they used libtorrent like everyone else. According to a comment on that issue it seems that not only does it not support v2, but it also breaks on opening v2 .torrent files. Lots of torrents are generated manually using qBittorrent, i.e. private trackers, this means that torrents might start breaking for transmission users soon.

Unless I'm mistaken, we'd need to settle on a "default" piece size[1] if we were to have automatic upgrading in a client. I learned recently[2] that there's no standard set by the protocol, so Client A might choose a 16KiB piece size, and Client B might take the same files and choose a 64KiB piece size, resulting in two different hashes (v1 or v2!) and thus two peer pools for the same file set.

1. https://stackoverflow.com/a/65299426/2700296

2. https://github.com/webtorrent/magnet-uri/pull/43#issuecommen...

If I read the article correctly in v2 each file will contain its own root hash, which is calculated by using 16KB "pieces", the piece size for the torrent creation is still used for data transmission between peers and integrity checks.

I'm not sure if the the piece side continues to have bearing when the hashes are all in a Merkle tree on uniform 16k blocks.

"Enforcing these encodings also help make it more likely that two people creating a torrent from the same files, end up with the same info-hash and the same torrent."

In the discussion of backwards compatibility, it appears that v1 collections of identical content encoded with different piece sizes will all become the same v2 collection when upgraded.

If you have a v1 torrent file, you can’t download from a upgraded client

The release of qBitorrent 4.4 has had an impressive impact. BitTorrent went from something like an average of 10 or so BitTorrent v2 hybrid torrents to 250 or so.

Granted, that's something like 0.4% of the new torrents made every day, but still, it's not a rounding error anymore.

[1] https://imgur.com/a/mPcEaT5

Which tracker is this data from?

It's from a DHT/BitTorrent crawler that runs around downloading all the torrent metadata it can find I've been running for ages.

Do you make visualizations of the traffic?

Some different views. For instance, to give some sense of a scale for the 250 or so v2 torrents seen every day is, here's a view of the total number of torrents it discovers each day [1]. (Or, more properly, this is the number of torrents it gathered complete metadata and validated against the infohash each day).

One of these days I should find somewhere to put all these bits of metadata I've been downloading.

[1] https://imgur.com/a/8pWhjxD

Consider creating a collection at the Internet Archive for these data sets.

Sure:

https://archive.org/details/torrent_metadata_archive_sample

I'll start uploading monthly archives.

I genuinely appreciate that you took the time to do this. Thank you.

For anyone looking to download the data: Click on my name in archive.org to see the monthly archives. I'm slowly working my way through the months, though there may be breaks in the work as other work takes priority.

The first few months will be the biggest, then it should settle down to something like 15-30gb a month.

Neat! I don’t have it handy, but years ago I scrapped the states traffic page and then made time lapses from them. One of the road segments had a bad sensor that would toggle into a stuck state.

Keep it up!

cool. Did you write it yourself or are you using an open source project of some kind? Would be interested to run one of these myself.

I stumbled across a DHT crawler, and started running it. Over time I added random things to it, like IPv6 and the ability to download metadata. I was mostly interested in using it to learn how DHT worked both in theory and in practice, and for that, it did it's job.

The page for the crawler I started with is long gone, but this appears to be a reasonable copy on github:

https://github.com/zlzlovezl/simDHT-1

I'm sure you can find other copies, with features added. Looking around, it seems like quite a few people have taken this and made it their own over time.

It's far from perfect in this state, but a fun toy to play with. I will say, from a point of painful experience: The way this crawler works tends to make some of the various people that track such things think you're sharing pretty much all the things. If that's an issue where you live, don't run it from your home connection.

I'd like qbittorrent to work, personally. Worked for 1 torrent, and now won't even contact trackers, just stalls. Installed deluge beta and works perfectly.

Reply on news.ycombinator.com