BitTorrent Turns 20: The File-Sharing Revolution Revisited | Hacker News Reader

BitTorrent Turns 20: The File-Sharing Revolution Revisited

torrentfreak.com

171 pointsboramalper90 comments

In my opinion the BitTorrent network, along with the DHT network, holds so much untapped potential. It’s a sea of constantly-changing data.

The data is so much more easily shareable compared to HTTP because all you need is a computer and an network connection. No need for a public IP address.

Where are the Googles or the Yahoos that harvest this data and make it easily accessible? If these companies made fortunes off the data that we put on HTTP sites (along with all the hurdles that populating a site with data comes with) then imagine how much more potential there is in the BitTorrent market where one can for instance very easily share any kind of info (albeit they must be online at first).

Sure now the network is populated by illegal movies and such. But if we populate it with blogs, recipe sites, documentations, videos, etc. a new kind of web can reborn that is more under our control.

Torrents are static and so are their hashes on the DHT. For blogs, recipe sites and say content creators putting a new video on their site every week, this is not very practical esp. when there are no peers to host their content if their computer is offline. IPFS has a similar problem.

Honestly, I could see this as a pitch for a decent tiktok like service. Your video lives as long as someone somewhere has watched it recently. I might actually enjoy something like that.

Whoa no way I was thinking of something similar, viral-only videos kind of like the Reddit-button https://en.m.wikipedia.org/wiki/The_Button_(Reddit)

It exists: https://joinpeertube.org/

The online peers are virtually servers. If a market is created then peers could get paid to host/seed your site. Instead of buying say a server as we do today for HTTP sites. Also there are mutable torrents.

> For blogs, recipe sites and say content creators putting a new video on their site every week, this is not very practical

IPNS can be used to leverage IPFS for dynamic or updatable content. Lack of peers is still an issue of course, but it does away with the need for a dedicated CDN once your content gets popular enough since users will be seeding it themselves.

That’s where the innovation is needed. You need some form of dedicated hosting, dns, a notification system to alert you when new content is available or automatically download it. The static nature of the content isn’t problem, that’s how media has existed forever and most web media is still only published once and never revised. With a smart enough client though you could receive patches to existing content. The space is more similar to a newsletter but much more capable.

Zeronet can serve frequently updated content in distributed (p2p) network. It works for blogs, forum and social network.

This is so spot on.

It's been some years I've been thinking that given that nowadays it doesn't take much effort to scan all the internet (masscan, etc), it might not be hard to find all trackers and then crawl all the DHT.

I'm not sure if this is feasible, but it might be an interesting start.

I've played around a bit with DHT indexing recently and a very simple python program using libtorrent to send sample_infohashes (BEP51) and download metadata (to get names/files) was enough to get me 1-2 .torrent files per second without any special effort or aggressive settings. The bottleneck (by 10x) has been the embarrassingly parallel info hash to .torrent step, so speeding things up shouldn't be very hard.

After running it sporadically for a few months I ended up with 1.4M torrent names and 30M info hashes, but I never put any work into estimating the size of the DHT, so I don't know what sort of coverage that represents.

If bittorrent was invented today, it would probably be tied to a cryptocurrency scheme.

Funny you mention that. The altcoin Tronix acquired BitTorrent a few years ago. I didn’t even know BitTorrent was a company until that happened.

How did a coin acquire a company? Or is there a company that issues Tronix that acquired the company?

As someone who uses private trackers, bear with me for a moment:

On private trackers, your download and upload is tracked. You are required to maintain a certain ratio.

Most tracker sites allow users to "donate" money to get extra ratio.

Most tracker sites allow users to request new files and pay a bounty, in uploads (i.e. ratio).

In effect, it is already a currency.

Trackers get seized from time to time, and if the entire state of a tracker is stored in a decentralised manner, then seizure is no longer a risk.

The dataset stored by a tracker, in the age of magnet links, is very small, and can be replicated amongst all nodes.

So, as someone who usually rolls my eyes at people saying "blockchain", this is actually a perfect use for blockchain technology.

Mutable torrents can be used for distributing a tracker’s metadata corpus using BitTorrent:

http://www.bittorrent.org/beps/bep_0046.html

https://news.ycombinator.com/item?id=12257065

https://news.ycombinator.com/item?id=17306106

https://static.usenix.org/events/woot10/tech/full_papers/Wol...

This is cool, but it doesn't address the user currency tracking that all private trackers have (so torrents reliably get seeded).

The problem of how to solve how much currency each pseudonym has, and allows transactions between them, with no central intermediary or point of failure, is solved by a cryptocurrency.

What about the economics of this "proof of seeding" scheme?

But wouldn't a public Blockchain make the private tracker not private anymore?

Private as in not visible, or private as in controlled by a single entity? If the former, just encrypt the data. If the latter, then use a smart contract where only certain key holders can update the tracker data.

This brought back fond memories of spending much of my teens on what.cd, good times.

> seizure is no longer a risk

Rather you mean, the ledger will be perfect evidence against all the users. Not sure why you need durability for “illegal” activity data?

Privacy friendly ledgers are old hat at this point, any hypothetical system for torrents would no doubt use one.

Proof of seed hasn’t been named yet.

It’s perfect except doesn’t require blockchain. It works well as just a simple semi-distributed database with sources of truth for who gets credit.

The trackers I’ve used are all different and a ratio on one site might be good to “vouch” for me, but the numbers are meaningless across sites and I don’t think one site would prosper mixing their ratio calculations with another.

I like your idea though and think that tracker ratios are a good example of something that meets the need that lots of coins are trying to accomplish.

Bram Cohen has not let you down! He makes coins now: https://www.chia.net/

What an odd viewpoint.

No wonder most people don't deliver amazing things - it avoids the social pressure to keep producing, lest we believe that person is letting us down.

BTW, the article includes: "BitTorrent’s inventor is one of the driving forces behind the ‘greener’ Chia coin, which launched earlier last May."

A coin that uses proof of storage space instead of cpu cycles is very debatable on the green creds.

Which is why I believe that 'greener' is in quotes.

But the main point of my heavily down-voted comment is that I can't conceive why anyone should feel let down by Cohen, even if he hadn't done a thing in the last 15 years.

I actually believe that view corrosively discourages people from doing something exceptional, for worry that they won't be able to maintain that output, and thus face social stigmas like "you let us down."

Sorry: "not let you down" was intended as a sort of joke. It should be read as the very opposite: Bram Cohen is basically a sellout capitalizing on his name-brand-value by making his next-gen distributed protocol tied to a token sale that will make him very rich.

Ahh, I see. I mis-interpreted it as comment in support of that particular form of cryptocurrency.

Thank you for the clarification!

Well, that happened eventually :)

https://www.bittorrent.com/token/btt/

Shame it happened, it ruined trust in the company when most employees left https://globalcoinreport.com/tron-trx-bittorrent-acquisition...

Fortunately BitTorrent the company isn't BitTorrent the protocol. I stopped using the "official" client a long time ago as it grew cancerous.

Bran Cohen himself got into cryptocurrency with Chia (which is however otherwise unrelated to BitTorrent), and BitTorrent itself got swallowed into TRON.

So yeah, it is already tied to a cryptocurrency scheme.

People have already made it into one

https://www.coindesk.com/price/bittorrent

It's annoying because I want to look up technical details of BitTorrent and have to now scrub past all these crypto fanatics and their posts about it going to the moon or crashing.

This is missing an important part of the story.

In the early 2000s slashdot.org was similar to what HN is now, with HN having more of a startup vibe and Slashdot having more Linux news. But a very similar audience, checked every day by them.

BitTorrent needed a very large scale test. They also needed a group of people technical enough to format BitTorrent and test it, and content he could license (ie that wasn’t pirated) to test it that would appeal to them.

So a post appeared on Slashdot: “Free porn”. BitTorrent licensed some very high resolution (way higher than most online video at the time) lesbian porn from lightspeed. All people had to do was install this tool called BitTorrent to get it.

‘To format BitTorrent’ -> ‘to install BitTorrent’

Is the torrent still up?

Was libtorrent v2 [0] finally adopted by enough clients and trackers to be used ?

I never encountered a magnet link including the "urn:btmh:" (instead of "urn:btih:" for v1), specific to hybrid or full libtorrent v2 torrents.

[0] https://blog.libtorrent.org/2020/09/bittorrent-v2/

I read an interview somewhere few years back that the original BitTorrent was written in Python using an editor called NEdit.

http://bittorrent.cvs.sourceforge.net/viewvc/bittorrent/?hid...

Like IRC, Bittorrent lacks convenience and a nice UX which seems to be required for mass adoption. Most people use dropbox for transferring large files. Some even use email as file transfer if it's below some maximum size (20-50 MB).

I have never actually created a torrent and I use the technology mostly for grabbing debian installer images etc.

I find both IRC and bittorrent to have a superb UX. It's minimal, highly interoperable and programmatically accessible. I've had sketchy experiences getting a connection between two specific peers though. I presumed it was NAT traversal being finnicky.

> I find both IRC and bittorrent to have a superb UX. It's minimal, highly interoperable and programmatically accessible.

I agree, but there's no way my mom could figure out IRC on her own. She uses Dropbox daily, though.

Before the WWW ordinary people did figure out IRC. It's just that people have got used to a different kind of UX - arguably what we have now is more complicated and harder to understand.

Yeah when I was in middle school in the late 90's all the normal girls hung out on IRC using mIRC. What killed IRC was more mobile I think, since once you need to set up a bouncer and stuff it actually gets difficult.

Anyone who can use a web browser can use IRC.

E.g.

https://kiwiirc.com/nextclient/irc.libera.chat:+6697#irc

https://web.libera.chat/?channels=#irc

There are also mobile apps with better persistence and even simpler interfaces.

IRC will never take over the world because:

* The protocol is missing some minor but important UX-enhancing features. This is fixable, but there's nothing valuable enough about the existing userbase for a capitalized entity to bother working with the entrenched groups. And non-capitalized entities move relatively slowly.

* There is no profit in it -- the corps who want to own messaging want to own messaging. This is a great idea for them, but a terrible idea for users.

* IRC networks don't want Eternal September to happen to them. Not that anyone is beating down their door.

> IRC networks don't want Eternal September to happen to them. Not that anyone is beating down their door.

Too late for that. In late 90's/early 00's, you had web clients for IRC.

IRC is missing many features that people expect these days though, whereas I don't think Bittorrent is. Bittorrent has definitely done a better job at evolving than IRC has too. E.g. with the DHT, magnet links, webtorrent, etc.

IRC doesn't even support basic text formatting, let alone links, images, emojis, threads, etc.

You're far from wrong, but I would quibble with a couple of points. Specifically, IRC does support basic text formatting; I've written a client library [1] to implement it. It's not very comprehensive or at all well specified, though. Some clients can handle emoji, links, and images, but it is client-dependent.

[1] https://github.com/aaron-em/rcirc-styles.el

> I've had sketchy experiences getting a connection between two specific peers though. I presumed it was NAT traversal being finnicky.

And that is probably are IPv4's flaws, not BitTorrent. If everyone switched to IPv6 then they can connect to each other without using NAT, because IPv6 address space is so big everybody on earth can have their own unique IP address.

People would still have to open a hole in their firewall, and then we’re back to people implementing UPnP

At least on IPv6 people know it's caused by their firewall, which they can fix on their own.

On IPv4 you don't know if your packets got dropped because of your firewall blocking some ports, or because ISP's CGNAT get overloaded and decide to drop your packets (and customers can't do anything to fix that)

>Bittorrent lacks convenience and a nice UX which seems to be required for mass adoption.

I thought it did have mass adoption.

My sister uses it, my Grandpa uses it, and pretty much every single person who watched Game of Thrones back in the day used it too.

(There was no way for Australians to watch Game of Thrones legally, the same day it came out in the US, until maybe season 3. Even then it was linked to a $90/month cable TV plan in a country with low cable adoption rates. Hence everybody torrented it and/or shared it with their workmates on USB sticks.)

Plenty of people just use putlocker or similar sites when pirating rather than torrents. Especially where I am (Germany) since fines for torrenting are extremely common.

Germans should vote out the politicians that allow this crap.

All German parties with even a slight adoption rate want this, not just via lobbying. There's no choice.

It's also not a "deal breaker" for a lot of people, same as with recent privacy-unfriendly laws(Bundestrojaner, Vorratsdatenspeicherung, etc).

Are they actually common? I hear stories about it here, but have never come across it IRL.

The stories of people having their access limited mostly seems to be PR from industry groups and VPN companies.

I know multiple people who've gotten one personally. They are perhaps a bit less common now as less people torrent without a VPN.

Fair enough. I guess it's different in Germany, then.

I got burned twice by this. Once for some shitty music and years later for a mediocre tv series I could have just watched on my Amazon TV stick instead. The first time I got away by just claiming it wasn't me and promising I'd make sure to prevent any copyright infringement from my connection in the future. There are lawyers who don't do anything else but scan bittorrent traffic all day and shotgun these threats. Mostly from Munich.

Games are free game though, also anything from Usenet is fine. Nowadays you could probably get away with a free ipv6 tunnel too, since adoption is high enough.

Loyal Pirate Gary voter here. Arrrrrr.

This feels like a troll post.

Neither IRC, nor BitTorrent have anything to do Dropbox. And both were incredibly popular back in the day

People use Dropbox because you don't need to open ports

The situation would be different if browsers had it built-in.

Perhaps it could be mutually beneficial. If Firefox had a Bittorrent client, both could gain popularity?

I never understood why browsers don’t natively support torrents so one can just <img src=“torrent://…”/>.

Opera did.

Oh, it's actually very easy to create a torrent, and I used it multiple times to transfer files for, say, a LAN party (games and/or patches).

It works much better with a tracker, though. Webtorrent is quite promising for the future, I believe you are looking for something like https://wormhole.app/ ?

It has a few features that other sites lack: resumable uploads, the recipient can start downloading before uploading is finished, no maximum file size (above 5 GB the file isn't stored on the server anymore this is just p2p and need to leave your browser open).

Amazing that not a single comment mentioned that BitTorrent was made for sharing anime originally and it's first users were all basically fansub groups.

I thought it was originally used to distribute Linux ISO files on release day when FTP servers were drowned in requests. I remember it was a Slashdot thread where people were for the first time trying it and being amazed at the speeds.

I believe it's claimed to be originally designed/used by a music sharing community (etree.org) where artists explicitly gave permission.

If you search for anonymous torrenting, check out I2P. This anonymous distributed network includes BitTorrent natively with many trackers.

If there is any file sharing protocol that has so many good memories tied to it in the last 14-16 odd years of using, bittorrent it is.

Selling BitTorrent to Justin sun was a big mistake...

About to sign up for Nord VPN in quest of obtaining some ultra rare bob dylan bootlegs not even sold in most record stores

Does anyone have a fav go-to source for torrenting these days? My last longterm success torrenting was with Demonoid but haven’t used it in the past 5 years or so.

Any feedback super appreciated

Orpheus is very good but you will need reputation somewhere or sit and wait on IRC for a while

Also a shout-out for my preferred VPN, Mullvad

Rutracker has everything.

IPv4 and nat is not the only problem that limits p2p, asymmetrically slower upload speeds , even on fiber are another problem.

I'd say a large part of BitTorrent's success was the mix of capitalism and communism.

To maximise your speed you had to share, a mini stock-market on each torrent. Then you were encouraged to continue to share after completion.

From memory all the fancy algorithms for sharing didn't really do much and it was just push it out and pull it in as fast as you can that really worked in practice.

It was also interesting Bram opposed encryption on the protocol.

The global torrent library certainly seems bigger than ever. The biggest issue is upgrading existing media to better resolutions.

I don't understand the importance of encryption in BitTorrent. Correct me if I'm wrong but sharing parties have no way of verifying each other's identity so the whole exchange is vurnable to MitM attacks. Also, file integrity is solved by the torrent metadata you add to the client.

I guess it makes it more costly to monitor content as the evil party has to involve itself instead of just monitoring.

My recollection is BitTorrent encryption has mostly nothing to do with real security, and almost everything to do with the cat-and-mouse against ISP middleboxes. Plaintext BitTorrent traffic was highly identifiable and blockable, but even simple obfuscation was enough to avoid them, so simple encryption was adopted.

Yes. It was mostly to stop ISP throttling.

Bram's case against was based on the idea ISPs couldn't cache data if you did.

Now we know encryption good, combined with things like peerblock it has a ok use. And the caching never happened.

Also at the time Skype was the spies messager because encryption was turned on by default, back then people looked suspicious if they encrypted, the extra encrypted data flowing around would have been good.

Bit-torrent was a failure in my opinion.

I really wish there was a client / protocol that allowed for sharing of files, but remove control from the user for what to share. Also limit the bandwidth on a monthly basis.

Sort of like a global peer-to-peer cloud. It could be used by open source organizations for sharing software and allow everyone to contribute to open source indirectly.

Users can designate a fixed size and bandwidth. Then whenever they use their computers, torrents will be running in the background.

This has a lot of issues for the user:

• their disk space is now taken up by arbitrary files they don't want and can't delete.

• some (if not a vast majority) of these files will be child pornography.

There would be benefits too though. CDNs would be marginally less useful lol.

Why would I want to relinquish control from what I share? Sounds like a liability nightmare.

Not in most western countries. It would be no different than running Tor nodes.

Bittorrent is more of a liability nightmare since it's super hard to block on a public network yet easy to record what IPs download certain torrents.

Running tor exit nodes is a legal liability in many western countries already, so "it's only as risky as tor" is small consolation. You might not get convicted ultimately but the process of explaining why your IP showed up on the access logs of the child porn site they just raided is a problematic enough prospect to scare most people off running an exit node

https://freenetproject.org/index.html

There's Freenet[1], but I don't think that ever got adoption to the level of bittorrent. But they have different goals and different uses. Bittorrent isn't intended to be a way to disrupt censorship; it's a way to distribute data at lower cost if some downloaders are willing to also upload, and there's some amount of reliability benefit as well in that a disappearing source can potentially be replaced by peers who got partial downloads, and it's successful at that.

If we didn't tend to have 100:1 download:upload ratios on our residential connections, it could have revolutionized more things, but it's effective in certain communities and has business applications too.

[1] https://en.m.wikipedia.org/wiki/Freenet

That's exactly how the Japanese filesharing scene developed - there's 3 proprietary programs that do just that (Winny, Share, Perfect Dark). Torrents aren't that popular there, and Japanese content torrent uploads aren't usually done by Japanese people.

This is how the technology from which Bram derived the core of the BitTorrent architecture (MojoNation, where he worked prior to releasing BitTorrent) worked, and it was a failure. When storage is abstracted away from the publisher in this manner it gets flooded with content that is so nasty no one wants to host it themselves (child porn, etc), when the bandwidth and storage accounting is split from the specific content being shared you end up with a lot more people trying to game the system because credit earned from one node or piece of content can be used for something you really want, and when you have a global cloud of p2p nodes the power is concentrated in the search engine necessary to find content and you end up even more centralized than the web is today.

Been there. Done that. It doesn't end up working out the way you think it should.

Reply on news.ycombinator.com