Hacker News Reader

Just be careful - unless you mark that torrent private it'll get posted to the DHT and crawlers like BtDig will pick it up and list it publicly.

For this reason I prefer using something like Syncthing which is designed more with this purpose in mind.

You could also gpg encrypt the file first which is a little more work but bulletproof. Maybe someone should make a tool that creates a torrent and encrypts it for you and presents a password box on the other end.

Like magic warmhole?

Magic Wormhole

https://magic-wormhole.readthedocs.io/en/latest

How are BtDig crawling the DHT? I thought this was made impossible or at least more difficult a while ago.

I don't see how it was made impossible. It's definitely possible, you can do it yourself there's many examples on github (even btdig has an old version on it)

Its trivial to disable PEX and DHT in any reasonable client

Sort of like Limewire sharing your documents directory?

They have MuWire now: https://muwire.com/

Not private but public sharing.

I am working on a tool that will help with this. It is built with AlpineJS, Nim, Aria2 and Webview resulting in a 5MB download which doesn't include the torrent file as that varies. The idea is that it is a one click solution as the torrent is embedded with the binary.

The inspiration for this tool is for assist with LAN parties. The one thing I have against the private flag is that it also disables LAN peer discovery, which would be okay if you use a tracker behind the LAN (though the torrent would need to be modified if the hostname/IP changes). Since you can't configure other people's clients I found it simpler to use Aria2 that is preconfigured to disable PEX and DHT.

Combine this with Metalink or Web Seeds, so that you can have a initial seeder based on HTTP. I think using IPFS would be a great web seed as long as the their gateway and Cloudfare's continues to stay up. IPFS creates a permanent URL if you will, so no need to worry about dynamic IPs or domains. It would be great if IPFS would have a smaller binary (right now it is around 20MB compressed) and had a way to get a file and "seed" it like a torrent. But for now I think torrent is mature enough.

Why not use Syncthing or Resilio Sync for LAN parties?

I have considered it. The one thing about Syncthing is that there is no true readonly way to sync. For example, someone may accidentally extract an archive in the Syncthing folder and it will sync everywhere. If Syncthing gets this feature I think I would be 100% onboard with it for LAN parties.

Resilio I believe supports it but I would prefer an opensource alternative.

https://github.com/syncthing/syncthing/issues/62

You can use the send only feature in Syncthing to make your folder read only.

https://docs.syncthing.net/users/foldertypes.html#folder-sen...

Sure, I can have for example my computer be send only. But unless everyone else configures it properly as receive only, they can start pushing data to other clients. It would be far easier to have Syncthing create a readonly share similar to Librevault and Resilio.

Technically speaking, what occurs when you mark a torrent private?

Does it actually block trackers from picking it up or just like tell them not to if they want to obey it?

I'm not an expert on this, but as far as I'm aware, the main difference is that your client won't go out and announce the infohash to the DHT (used for trackerless torrents, magnet links, that kind of thing) and won't provide it to other clients for download. It'll only ever send requests relating to it to the trackers specifically listed within the torrent itself. This means that anyone who wants to download it basically needs to posses the torrent file. Trackers don't really crawl in this way, however, some searching sites like the one I mentioned there do, they'll go out and look at the entire DHT, trying to find entries that they don't know yet and when they find one - go download the torrent and post it to their index.

For some reason I haven’t understood, syncthing would often miss transferring files but mark the entire transfer/sync as complete. No errors in the logs either. I tried it for a few months earlier this year, then gave up and switched to plain rsync in a shell. Based on that sample of one personal experience, I wouldn’t recommend syncthing to anyone.

Note: I was using syncthing for a one way sync all the time, with send-only and receive-only settings on the endpoints.

I've been using Syncthing extensively for years and never observed this behavior - it always either completes the transfer or fails to complete the transfer (which is obvious in the UI). A silent failure is certainly not typical.

This sounds like a pretty severe bug, so if you still have the systems you were able to produce the behavior on it might be helpful to reproduce it and submit an issue.

Yeah, I gave up an Syncthing when using it between my Android device and Linux server. Too many hours were lost trying to debug why the server no longer showed as connected on the phone. At some point I went back to emailing myself files, because that actually worked, and I realized that I didn't actually need an entire program with a synchronization algorithm to send a few large files on the occasion.

That's why you encrypt it or at least make it a password protected archive. That way no one will see the file names in the content list. And without seeders, it's highly unlikely anyone will download it. Torrents work well with 1 seeder, 1 downloader.

Sometimes I wonder if the bad connotation due to piracy made this protocol not standard (in a sense, that's not widely adopted, and people need dedicated clients, instead of it working for example in browsers).

I don't know enough, maybe it has security issues, or some limits... but the fact is that it's still widely used, by tech literate people, so it still stands the test of time and it must be doing something right.

> people need dedicated clients, instead of it working for example in browsers

"Version 9.0 [...], released on June 20, 2006" "Opera introduced [...] a built-in BitTorrent client" [1]

That's history now, of course, courtesy of today's browser monoculture.

[1] https://en.wikipedia.org/wiki/History_of_the_Opera_web_brows...

Does Firefox have an extension to download torernts?

I know it used to have one, but that now that I'm aware of the v57+ extension issue, I notice that the one linked to first in a search seems to be dead.

Brave has a built-in torrent downloader.

WebTorrent [1] can be used to run BitTorrent on the web. It's actually integrated into the Brave browser which is a nice feature. Other browsers might catch on soon.

[1] https://webtorrent.io/

WebTorrent is pretty limited though:

> ... a browser-based WebTorrent client or "web peer" can only connect to other clients that support WebTorrent/WebRTC.

(from https://webtorrent.io/faq)

This essentially splinters the world into the bittorrent and webtorrent halves, which makes it far less useful.

WebTorrent is part of libtorrent now, so should be part of all torrent clients based on it, e.g. qBittorrent:

https://github.com/arvidn/libtorrent/pull/4123

Not quite there yet. Latest release of libtorrent was 1.2.9, seems it's scheduled for "some branch" after 2.0 (https://github.com/arvidn/libtorrent/pull/4123#issuecomment-...). Add in some time for clients to actually start using it as well, which is not guaranteed either.

The DHT remains split for now.

For some reason browser vendors do not allow JavaScript to connect to any port. So there's no other way.

If you want a CLI/Desktop WebTorrent client https://github.com/anacrolix/torrent works great.

Caution: Don't open this page on a metered connection, immediately starts downloading a 130MB movie file.

Ouch, I understand that it demoes their solution, but with the autoplay that's very insensitive.

Probably, but it is quite a neat technology. As people in this industry, we know what else it is being used for, and it serves those purposes pretty well. I love torrents. I love the technologies that it uses. Many other projects use it, for example take a look at https://docs.ipfs.io/concepts/dht and https://github.com/savoirfairelinux/opendht just for DHT alone. There is also https://en.wikipedia.org/wiki/Magnet_URI_scheme. There are many more, just too lazy to find them at the moment, but believe me, the things you will encounter once you go down this road, is truly amazing. Worth it!

It was probably phones and datacaps that really did it. Torrents are really only ok on laptops/desktops and the unexpected data usage for customers is much worse than the costs of distributing data.

Steam could probably benefit loads from torrents but they do have extremely good distribution servers already.

BitTorrent didn't catch on simply because there aren't very many actual use cases for it. The majority of P2P content distribution will always be illegal (due to the nature of copyright), and so the client-server model is much more representative of the real world. It is technologically pretty neat though, and some publishers do actually use the protocol for things like software updates.

Not very many use cases??

Except for: - updates of software. - distribution of public video files. - distribution of install images. - distribution of raw public measurements (ea science data).

Basically BitTorrent is the protocol for any usecase with 1:n (or n:n) with public data (ea. No data access controle is needed).

Copyright has nothing to do with it and the nature of BitTorrent does make it also a good fit for some pirates.

Never forget that the following is shared over BitTorrent: - World of Warcraft updates. - Linux distribution install images. - haveibeenpowned sha lists.

And that’s just what I personally know. I know there is more.

In my previous job we used BitTorrent to distribute movies to cinemas. :-)

What an irony! :)

> the bad connotation due to piracy made this protocol not standard

Back when Resilio Sync was known as Bittorrent Sync, I proposed it as a file-sharing solution between multiple locations for the company I was working for. Immediately returned with an emphatic refusal to use any kind of software that had "Bittorrent" in the name, regardless of what it was for.

This is the same company where they picked up I had "Bittorrent Software" installed on my work laptop, and the people who ran the auditing software freaked out because, you know, it's Bittorrent! I told them it was a single icon in my PortableApps folder but they just kept saying "There's a Bittorrent client on your PC, you could be fired!" until I deleted the icon.

So yes, the fact it's associated with piracy has made a huge difference in some places.

I was kicked out of the public library once for running Syncthing, I guess because the firewall detected it as some sort of piracy tool

> Unlike HTTP, you get reasonable retry behavior on network hiccups

Seriously. Zmodem/resume was doing better in 1986.

wget -c, it ain't hard.

The problem isn't the clients.

The HTTP server needs to provide optional support for range requests on that resource. (RFC 7233) Not all do.

Every major webserver supports resume.

You need to notice the connection died and do that manually, or think ahead and have your script do it in a loop. A bt client already has the reconnect logic. And it will hash the contents to make sure it worked correctly.

I've written a cross platform gui tool to transfer arbitrary large files via p2p with end-to-end encryption.

https://cryonet.io

- Fully p2p, even the signalling server part which is done via Tor hidden services (later on there will be an alternative using a DHT).

- So unless the Internet or Tor goes offline it can never be down.

- Works with proxies and in corporate environments when UDP might be blocked.

I had to transfer 2tb of data over a wireless connection that kept dropping packets. BitTorrent was the only method I found that worked; everything else either broke down when it disconnected, or had too low of a retry limit to be useful, or wouldn't resume properly.

Can you recommend easy software to do the creation? Don't you need a tracker as well?

+1

Almost all BT clients can create torrents. You can use a public tracker if you want, or you can use a magnet link (DHT magic will find the peer eventually), or worst-case scenario you can manually add the peer IP on either end.

qBittorrent has a built in tracker for private torrents https://www.ghacks.net/2019/09/20/how-to-create-a-private-to... this method will also not leak information into the DHT though 99% of the time you don't care about that.

transmission is pretty complete and easy to use

With transmission-create you can create a torrent from the commandline. Load it into your bittorrent client, find its hash in the UI, and make it into a magnet link, like:

magnet:?xt=urn:btih:[this is where the hash goes]

Example:

magnet:?xt=urn:btih:a008663c4fc7e710bd59200e5ef99ba04f83f319

There's tons of available trackers. Just use whatever one you want. Also most clients can create torrents.

BitTorrent is the best for sending large files. I wish there existed a tool to make torrent creation easier, and I also wish my computer could seed. Every time I create a torrent, nobody can discover my server, even though the port is open and even if I add the server IP as a peer directly onto the client.

I have no trouble seeding in swarms, only when I create torrents. Other than that, BT is ideal.

Maybe you're only able to make outbound connections and not able to accept incoming connections? Try port scanning your PC from the internet

No, connecting externally is fine, that's what's so weird about it.

>Just encrypt your stuff beforehand.

This is such a facepalm moment. "This service is awesome, it does everything you need, it's reliable, it's safe, it's practical, and all you have to do is encrypt your files, share encryption keys, validate identities, and establish trust with the person you're sharing with. Also if you screw it up in any way, your file is public forever. It's so rad!"

No, "here's a magnet link, and here's the matching 50-char password; don't share the password with anyone you don't want have file". Just because you can make security more complicated doesn't mean you're obligated to.

> Also if you screw it up in any way, your file is public forever.

You can literally say that for ANYTHING out there. Not sure what point you are trying to make. Let me guess, "Encryption is hard?". Well there's no free lunch.

Well there was free lunch when Mozilla's Send thing was running. Sharing the password in the URL hash was brilliant. But then it ran into all the same Mega.nz problems where people immediately use it to host malware.

Encryption is hard, which is why it's sad to see that when someone gets it right and nails the UX, it still gets ruined by the real world somehow.