Show HN: YouTube Filesystem – YTFS | Hacker News Reader

Show HN: YouTube Filesystem – YTFS

github.com

185 pointsrasguanabana61 comments

Haha, from the title I expected this to be a solution for generating MP4s from arbitrary data that you could then upload to YouTube for backup.

This is definitely really cool, too. I like that it shows that a node in a filesystem is just another way to represent some resource, but in a way that's a little more familiar to most people than something like procfs.

That's a fascinating idea. Unlike gmail, there isn't an upper bound on how much you can store on YouTube. If you could encode data in a way that's resilient against re-encoding, you could use YouTube for steganography as well. Or, for that matter, see http://dataliberation.blogspot.com/2012/09/your-youtube-orig... .

That said, it's also amazingly wasteful, and abusing a free service for a purpose it isn't intended for.

Definitely. And you'd have no guarantee that your video wouldn't get transcoded someday into a format or setting that corrupts it.

Speaking of corruption, I found this[1] video a while back. On mobile the video shows up fine, but on desktop it's just a gray screen, although the thumbnails work.

[1]: https://www.youtube.com/watch?v=lOb8TNn7sDE

Also the original "Leroy Jenkins" video has been corrupted by youtube.

Here's another video that doesn't behave well on desktop: https://www.youtube.com/watch?v=dQw4w9WgXcQ

The video is showing fine here. I'm using the HTML Video Everywhere! extension.

I can confirm that the mp4 I downloaded is not (detectably by my desktop video player) corrupt. I am not sure what it is, but it is not corrupt. (Except for the filename, but that is because javascript is brain-damaged as regards strings.)

I also have cookies and javascript disabled. Maybe the javascript is messing with the video.

It's just a gray screen for me in both HTML5 and Flash players, though if I download the video and play it in MPC-HC it's fine. Weird stuff!

Unless what you were encoding wasn't meant to be consumed as a bytestream. If you encoded a resilient optical format (like UPC or QR codes), transcoding the format shouldn't be a deal breaker. Obviously it's not optimized for backing up a harddisk, though.

Edit: @ionforce hints at same here: https://news.ycombinator.com/item?id=9840838

Interesting - say video @ 60fps, encode 1 QR code per frame, would be highly resistant against transcoding errors, very easy to extract information from again given the standard format.

Wouldn't be terribly efficient though. Wikipedia says max bytes per code is 2953 per QR code [1]. So 2953Bps * 60fps = 177KBps data encoding. I guess that's what you get for encoding it in a visual (human readable) format instead of a datastream directly.

[1] https://en.wikipedia.org/wiki/QR_code#Storage

you have sound too. Must be an audio equivalent that would have a similar level of durability to a qr code. I dont know if youtube ever drops frames during compression. Perhaps using there 4k support would help get a bit more data

You could reference this[0] vintage Commodore tech for ideas.

[0] https://en.wikipedia.org/wiki/Commodore_Datasette

Have a really crappy, coarse-grained visual format. Dark squares to communicate bits.

Maybe using Fourier/wavelet/whatsoever transform would be the way to go, just like in digital watermarking techniques. Both high capacity and robustness would seem easier to achieve.

Interesting. I guess we can experiment with how many bits can be compressed to a video frame. Is there a guarantee that YT doesn't change the frame rate?

Another idea is YT as code repo: essentially one makes a movie that shows code files. On retrieval, OCR can be applied to transform the movie back to code in text.

I know that YouTube used to support only up to 30fps video, but IIRC they now support 60fps. This became a thing because people making videos of themselves playing the newest generation of consoles (PS4 / XBOne) want to upload in high quality, and the consoles now do 1080p at 60fps.

If this is a concern for people (recording at 60fps to upload for 60fps), I doubt that Google would downgrade the framerate except for maybe the lower quality versions of the video (does 60fps really matter for 240p video?).

Can't you always download the original file if you're the uploader?

Until they catch on and disallow it, sure :)

They won't even take the upload, so we need not worry about the transcoding part.

trying to upload a pdf file : "The video has failed to process. Please make sure you are uploading a supported file type."

What they meant was taking arbitrary data and turning it into a valid video file that plays back. For instance, you could read each bit off as audio (zero, one, zero, zero...). The quest is to find the most efficient, yet resilient way to do so.

They said they would like to have some encoding technique (completely new) which would get transcoded without any data loss. So, my point was that such _new_ encoding techniques will be rejected by YT in the first step itself, before even transcoding.

No, they mean new encoding within the video and audio. A watermark is encoded in video, even though it's just visual data. Encoding can mean different things at different levels.

I've toyed with the idea of it before- it's actually pretty easy to transcode files into images, and if you were more clever than I you could probably figure out how to use color to represent larger chunks of data instead of just black and white squares. It's also not size friendly mind you. The files I generated were about 2x bigger than the original file.

If there was a way to do massive changes in existing videos it would be feasible to use them as storage - that would be a definitely interesting approach. Anyway, advanced data encoding would be needed to prevent it from damage after format changes. Well, it's not impossible and even not that hard to achieve.

And if you share your files (videos) you'd generate some revenue too.

Not sure if this ever went anywhere but this came to mind. https://secure.graduateschool.vt.edu/GSITWiki/Wiki.jsp?page=...

Glad I'm not the only one! Cause that would be friggin awesome haha

From the title, I expected this to be the filesystem used by Youtube.

Interesting! Looks like this is youtube-dl combined with FUSE, so that you can use a normal video player to play and filesystem tools to browse, rather than downloading first. See https://github.com/rasguanabana/ytfs/blob/master/doc/depende... .

This is great! Also, check out this easter egg which makes every video rick roll haha: https://github.com/rasguanabana/ytfs/blob/master/stor.py#L12...

Neat! Any plans to make the different video qualities quickly available? Maybe as a subdir named with the resolution, eg search/movie/480p.mp4 or maybe movie.1080p.mp4?

Oh, I don't want to clutter it too much, it's better to keep it simple.

It definitely needs a mount option to set the default quality. Changing it later might be tricky, maybe passing options in directory names would be a good idea? for example: mkdir "foobar [q:720p]"

How about doing it with mount options, so someone would mount /mnt/youtube/hires /mnt/youtube/lowres with different options?

That is going to be the first thing to implement now.

It seems not to support streaming. It has to download the whole video before playing can begin.

Some time ago Youtube started to provide audio and video separately. Due to that, for full videos whole data needs to be downloaded before reading (it's hard to merge them on-the-fly). For just audio or just video data streaming is intended, but needs reimplementation.

Are you sure ? youtube-dl still gives link to plain old streamable mp4 (I don't know much, I just use it half blindly).

I think these monolithic files are provided only for up to 720p quality, mainly for compatibility with older Flash players. To download the maximum quality with youtube-dl, you have to use "-f bestvideo+bestaudio" - then, youtube-dl will download both streams, and merge them for you with an ffmpeg invocation.

Ha, I never paid much attention to higher bitrates, and most of the time I only see resolution up to 720p (maybe because of my tastes).

I'm sure there are ways to merge streams on the fly though.

There has to be some way, the player on the YouTube website does it. :)

yes. you can force the streamable mpg4 download with the -f option (-f22), but it seems not all videos are encoded that way. livestreamer might be the tool of choice if you don't want to use your browser to watch a youtube video.

Are there any installations instructions? This seems to depend on a lot of libraries that are not specified anywhere.

The world would be a better place if we could run "rm -rf ./comments/*" on YouTube.

Theres a chrome extension called alientube that shows reddit comments instead (useful for finding obscure niche subreddits too). Or there's plenty of extensions to hide comments.

My favorite is 'Herp Derp for YouTube'.

It changes every comment to a mix of

> Derp herpy derp derp

while still allowing you to click and see the "quality" comment that the person left.

https://chrome.google.com/webstore/detail/herp-derp-for-yout...

The file system is a great UI that doesn't pretend to be a great UI. Thanks!

kudos, that's an excellent idea, combining out-of-the box thinking, data structuring, creating value all using basic tools. A tip to the hat. Nice work!

Might I recommend not using the prev/next executables and instead adding another directory level to indicate search page, e.g. ytfs/search/1/video.mp4

would make it easy to reexport this file system, e.g. Over http or smb.

I have to think that through. Search pages are tricky as you can't simply go to a certain page. They're identified by tokens, which aren't known a priori - you just get adjacent ones with search results.

A reasonable solution would be ability to set max results and disabling next/prev. All desired results would show up in the search directory (if you can assume that results beyond some point are useless).

I think the most obvious way to expose that remote behaviour in the filesystem is to have the directory /search/2/ appear only after the directory /search/1/ has been stat'd.

Does this use a persistent cookie which YouTube can use to personalize search results?

There's no authorisation at the moment. Plain requests.get is used.

side-note: I find it very very interesting to think about a filesystem of unknown content and size. It's basically an infinite tree through FUSE. Nothing crazy, but interacting with it directly is inspiring.

Back in the kernel 2.2 days there was an explosion of experimental filesysyems, like ftpfs, cdfs/cdripping fs, and more. Now that we have widespread FUSE bindings it seems we'll see a filesystem renaissance.

Right but ftp/cd are finite data sets. So far nobody has access to youtube full index and the virtual fs has to generate/navigate on the fly. Imagine a GoogleSearchFS, makes things a little different.

how is it a filesystem? can I store a PDF or zip file on youtube with that ?

It's read-only - you can play/copy youtube movies as a file

Storing a pdf or zip file is not a requirement nor definition of a filesystem. You can do neither of those with a CDROM or DVD and both those contain filesystems.

I'd imagine it's read only, much like parts of procfs.

Reply on news.ycombinator.com