Comment by dcposch | Hacker News Reader

Comment by dcposch | Hacker News Reader

This is what Feross and I have been working on the past few weeks. It's been a lot of fun. Here are a few of the things I learned:

* Electron is surprisingly nice to work with. It is simple, pragmatic, and thorough. System tray integrations, desktop notifications, launcher progress bar in OSX, installers for all three platforms, seamless auto updaters for Mac and Windows, a crash reporter, and so on --- all built in. Many of the little things that an app needs to feel native and professional have clean cross-platform APIs.

* Shout out to Cheng Zhao, aka zcbenz. He's incredibly prolific and seems to be nearly single-handedly developing the Electron project. Every single page of Electron docs or code I've looked at on Github, the latest commit on it is zcbenz.

* Peer-to-peer is moving very fast. WebRTC support in browsers was flaky and buggy until very recently. Even today, Chrome has a few big bugs left ( eg https://bugs.chromium.org/p/chromium/issues/detail?id=392651 ) and WebRTC is not yet supported in Web Workers ( https://bugs.chromium.org/p/chromium/issues/detail?id=302019 ). But things are improving quickly. Edge is getting Data Channel support soon. Safari is almost certainly gonna get WebRTC support as well -- it's already in WebKit, just not enabled in Safari yet.

Tons of cool new uses of the web are just at the edge of becoming possible.

I'm really excited about the way WebTorrent Desktop bridges the WebTorrent and BitTorrent networks. If enough people end up running it, BitTorrent in the browser will really happen. Projects like the Internet Archive can make huge files available for immediate streaming on their website, without paying for bandwidth. It's going to be interesting times.

One more:

* It's surprisingly fun and liberating working on a completely noncommercial project.

I have no problem at all with commercial software -- especially if it's free of ads and "microtransactions" and is simply sold to the user for a clear up-front price.

That said, it's fun just shipping something that anybody can work on, with no boss, no formal user requirements, no worries about whether the app will sell or whether the client will pay you.

The only goal here is to make something we're proud of and which users enjoy.

And... one more:

* It's really fun to share project ownership with all the contributors.

From CONTRIBUTING.md:

> Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project. (https://github.com/feross/webtorrent-desktop/blob/master/CON...)

The first time I witnessed this in action was on the LevelUp project (https://github.com/Level/levelup). Since the days of the iojs fork, Node.js itself has adopted the same model.

Sharing commit access liberally is awesome for project health, because:

- The project can move forward even when the project owner is busy.

- It empowers contributors. It makes them feel like owners, because, well, they are. They end up contributing a lot more code.

- In the worst cast, it's Git, so bad commits can be backed out. But I've NEVER had to do this, or ever seen anyone else need to do it.

hey there! One feature that XBMC (now called Kodi) had since 2004 but I haven't seen anywhere else, is the ability to play the contents of rar files.

So you could download movies and never have to bother unzipping them.

It worked for multipart rars too.

The lore is that they implemented a custom rar algorithm but anyway, that was 14 years ago and its an open source project so I should see this somewhere else by now!

Since you guys are doing something over the torrent network where these kind of video releases are common (multipart rar, reminiscent of the release group's newsgroup roots), could you implement this functionality too:

When streaming, discover the sequence of the rar files in the torrent, prioritize the file downloads by rar number, and play the video file within?

This is an incredibly niche feature today because no releases come packaged in archives. Furthermore, to my knowledge you can't really "stream" archives.

They still do come packaged directly from scene releases. Smaller private torrent trackers have these releases. Public trackers usually contain just the media files so people don't have to unzip. But I agree, playing from archives would be a great feature.

I'd say the majority still are split into rars for the initial scene release but by the time it comes to the public trackers, and even many private trackers, someone has already unrared it for you.

I've implemented the same thing in a Usenet/NZB streaming project a few years ago - it's actually trivial since the multi-part rars are uncompressed, so the raw bytes of inner-content are there ready to be read. Just have to handle RAR headers/trailers. Seeking worked too since Usenet posts were commonly made up of 700kb posts, so if a seek occurred, you prioritize retrieving the required posts where the user seeked to, to fill the media player buffer. Code has never been released for obvious reasons :)

multipart rars are uncompressed? this whole time? my whole world is shattered right now

A bit of an aside, but for anyone who thinks this sort of thing is technically cool, Free Download Manager will let you look inside a remote ZIP file (of any size) and let you pick the files you want to download. Relatively simple to implement, but cute. It's the only open-source download manager offering this, I think.

(http://freedownloadmanager.org/ is a bit Softonic-esque, probably because of the program's heritage; the code's over at https://sourceforge.net/p/freedownload/code/HEAD/tree/trunc/, last checkin 2 weeks ago - and the project description says "we also plan to port it to Linux/OS X," which is cool)

I believe this can be achieved via a combination of httpfs [1] and archivemount [2]. Not sure about the reliability, but I think the approach is more general - you could even stream from within a remote archive this way, using whatever program you wanted. The job of an archive reader, disk reader, or "server reader" should be to translate these things into the filesystem so that all programs benefit. GNU Hurd took this approach, with the native ability to mount tarballs and FTP servers as filesystems, and POP3 as an mbox file.

Incidentally, I can't find any archives with audio hosted on a server that supports Accept-Ranges on the open web. I'd appreciate a link if anyone finds one.

Update: AVFS [3] seems to be an all-in-one solution that handles HTTP, FTP, WebDAV, Zip/RAR/Bzip, SSH, and others. Also see "podfuk" [4], though it looks pretty ancient.

[1] http://httpfs.sourceforge.net/ [2] http://www.cybernoia.de/software/archivemount/ [3] http://avf.sourceforge.net/ [4] http://atrey.karlin.mff.cuni.cz/~machek/podfuk/podfuk.html

Just wondering, because it is not explicitly mentioned in the docs, does it only stream or does it also seed the downloaded chunks?

It seeds, like all good torrent clients.

Yes. All torrent clients I know, including WebTorrent Desktop, seed while downloading. WebTorrent Desktop keeps seeding after you've finished downloading a file, and keeps running in the system tray after you close the window unless you explicitly tell it to quit.

We want to be good citizens and keep the swarms healthy.

Absolutely. I'm just asking because in certain jurisdictions seeding (illegal content) will lead to huge fines, and not mentioning this on the homepage might lead people to believe that they can safely use this software. This is especially worrying since "streaming" is oftentimes understood as a read-only operation which cannot be pursued by law enforcement.

The general rule of thumb that seems to keep you under the radar is to limit your upload rate. Most clients have an easy option to do this - although apparently this is non-trivial in this client[1]?

[1] https://github.com/feross/webtorrent/issues/163

I didn't realize zcbenz was involved. He was also a big contributor to node-webkit.

Is Adaptive Bitrate Streaming possible?

I would have thought not.

Well, it would be possible but only if the video format in the torrent allowed for this is a way compatible with the way torrent transfers are split into blocks - i.e. it has a block structure that lines up with the torrent block structure such that block X is enough to play a given set of frames at the lowest quality, block X+1 for better quality, and so forth.

The blocks wouldn't have to be sequential: as torrent transfers are essentially random access you could have the content required for lowest quality display at the start of the file and the extra data further on, similar to how FLIF supports almost immediate low quality with progressive improvement as the rest of the file arrives (http://flif.info/ - see the video demo part way down that page, or https://uprootlabs.github.io/poly-flif/ for a more interactive look at how it copes with truncated files).

I'm not aware of any video formats that are arranged this way though. Those that are designed for adaptive streaming have a more dynamic structure that would not align well with the torrent protocols blocks.

Adaptive Bitrate Streaming works by having multiple copies of the source video at different bitrates, resolutions, and audio qualities.

Torrents are usually one file, which would not allow Adaptive Bitrate Streaming.

Reply on news.ycombinator.com