Intermodal: A new command-line BitTorrent metainfo utility | Hacker News Reader

Intermodal: A new command-line BitTorrent metainfo utility

rodarmor.com

80 pointsrodarmor24 comments

A suggestion, since I see this more and more especially in communities that consist of (relatively) young people.

Don't use discord. Even if you don't agree with the spyware characterization, there are very good social reasons to choose something else (e.g. IRC/Freenode or Matrix or preferably both). I don't use or advise people use discord for anything, but I don't really have any strong opinions if gamers or youtubers/"social influencers" end up going that route (because to them it's a means to an end for their monetization strategies). But it's a totally different situation for open technical communities.

Going with discord you are basically isolating yourself and the community you are trying to form from the older generations. People who grew up on the Internet in the 90s and 2000s and for whom IRC is bread & butter. These are also the folks that have accumulated a lot of experience and wisdom and that you really, really, don't want to be apart from. Especially if you don't have time to waste in mistakes that others have done and learned from before you.

These "wise veterans" are not going to come to you, in discord, but you can go after them by signaling through setting up shop in the places I mentioned.

I think those are fair points.

Personally, I have found Matrix to be extremely buggy, and much less usable than Discord. And, I now have an IRC setup that works for me using IRCCloud, but that took a while and I pay $5 per month for it.

Also, although Discord somewhat isolates one from older generations, IRC or Matrix somewhat isolates one from a different group, and I think it would be hard to argue that one was more important than the other. I've already gotten a bunch of early use, along with a lot of feedback, from users on Discord.

But, I do use IRC, so I should definitely consider having a link to #intermodal on Freenode in the readme. I sort of worry about fragmenting the chat, and having to check a lot of chat clients tho :/

I just registered #intermodal on freenode. I'll also mention that in the readme.

Man, this looks slick! And I loved your rant regarding collecting vs tagging/sorting, definitely can relate. I miss my CDs :)

It's a pity that most users are with dynamic IP and behind a NAT. This makes sharing directly with someone via BT impractical, so people still use email/messengers for that.

Dynamic IPs are not a problem for torrents. As for NAT: Many routers support PCP or UPnP IGD which lets the torrent client open the port. µTP can also exploit from simple UDP hole punching to allow incoming connections if the NAT is full cone (alas, iptables' MASQUERADE is not, despite IETF recommendations).

So whether it works depends on more circumstances

What you say is correct, but note that I was talking about direct connection to a specific person.

BT is great for "giving out to the world", but if you want to send a file to a friend, using an email, a cloud drive link or Skype/whatever is more practical. Intermediary storage is required for asynchronous transmission, and even if both you and your friend are online, you still need to take an extra steps of creating the metainfo file and sending it or a magnet link first and then wait for the DHT announce and lookup procedures to execute, which can take quite some time.

Yeah, at best it would be for technical users, and even then for 1:1 sharing a streamlined client tailored to that usecase would probably help. E.g. various status indicators that show whether nat traveral/hole punching is possible, discovery of the public IP address to include it in the magnet link (the x.pe extension!) to skip the DHT lookup if possible and a less complex UI. If you do 1:1 transfer you don't really care about your share ratio for example.

Even using public trackers (in addition to DHT) I haven't had any luck trying to send friends my torrents. They just never discover my seed, even though I'm conectable. It's very odd.

If you are connectable you should set up your own tracker. Even the old python-based bittorrent command line utilities will work for this and it's a piece of cake to get going.

There’s not much point using BT for a one to one transfer. If no one is going to help, why bother?

Thank you!

A sort of no-setup BitTorrent-only proxy would be interesting, to avoid NAT issues.

Have you run this against a large collection of torrent files? There are many edge cases. E.g. many metadata handlers barf on very large torrents such as the TLMC or danbooru2019. Others have issues with non-utf8 filenames since while it is strongly recommended it's not required (since unix doesn't require it either).

I just downloaded the danbooru2019 metainfo files, and used `imdl torrent show` to view them, which worked. (Those are some big torrents!)

I don't support non-unicode filenames, which is pretty unfortunate. I have a hunch that it would be a little painful to add, and that time might be better spent on other features, but I could be convinced otherwise if there was a strong need!

As one of the few who still maintains a personal music collection I applaud the amount of thinking that has gone into the question of how to make doing so as frictionless for nontechnical users as Spotify.

I like the idea of a common metadata format but I wonder what the next step is after that. What problems will this solve that can't be solved by ID3 tags for MP3s, and what is the benefit of having a single metadata container for music and films?

I think the main benefit would be how much it would simplify software. Software that wants to do something useful with user-downloaded content doesn't have very good options.

To do more than just display a list of files, and maybe try to render a preview based on the file extension, software has to implement support for reading lots of different kinds of files and parsing lots of different kinds of metadata in those files. (MP3, MKV, PNG, JPG, OGG, etc).

And then, because the data contained in that metadata is only semi-structured, and varies between content types, it has to do a lot of error-prone heuristics to do anything useful with that metadata. Like, for example, figure out that these files are the tracks of an album, and this file is the cover art.

This is definitely possible! But it's very hard, and even then has a long tail of issues.

If a publisher or packager could add structured metadata with a constant format and filename to a release, then all the downstream software and services could mechanically consume that one source of metadata, which would simplify everything enormously.

It would be nice if it would have a .gitignore like ignore file. Or does it?

It tries to ignore common junk files, and it can be configured to ignore things when invoking it with `--glob !PATTERN`, but there isn't anything like a `.gitignore` file at the moment.

Were you thinking of a `.gitignore` file that would be global? Like `~/.config/imdl/ignore`, or on a per-torrent basis?

I'd like to add a config in the future, and adding files to ignore to the config could be useful.

I was thinking on a per torrent basis. The .gitignore format is well known and there are a lot of goot templates on github et al, so using them would be a very convenient way to share github managed directories via your tool.

That's a good idea. There's already a `--glob` option, which allows selecting what to include with globs. It's the inverse of `.gitignore`, so `--glob foo` includes foo, and `--glob !foo` ignores foo.

I think that the glob library I'm using implements the same syntax as `.gitignore`, so it would be easy to add an `--ignore-file PATH` option.

I opened an issue to track this: https://github.com/casey/intermodal/issues/378

A bit offtopic, but does anyone know how’s the adoption of BEP39 and BEP46 standards (for mutable torrents)?

libtorrent implements it, so most torrent clients should have it

Do clients actually expose that functionality? I haven't seen those BEPs being used in the wild, so I'm very curious of features using those BEPs are actually available to users in existing clients.

They don't need to enable it, it's enabled. Yes, you can put mutable data in the DHT used by all libtorrent programs (qBittorrent for instance).

I might be misunderstanding, but I would expect that both BEP 39 and BEP 46 would have to be exposed to the end user in the UI, and I don't see that in qBittorrent.

For BEP 39, "Updating Torrents Via Feed URL", added torrents with `info.update-url` would hit a feed to check for updates. Does qBittorrent actually do this?

For BEP 46, "Updating Torrents Via DHT Mutable Items", qBitorrent would have to allow adding torrents via magnet link with `?xs=urn:btpk:PUBKEY`. I tried this, and it didn't work. Is there some other way to do this?

Reply on news.ycombinator.com