Hacker News Reader

Comment by RodgerTheGreat | Hacker News Reader

For some time I've been tinkering with a script which manages my own fine-grained youtube subscription preferences, periodically scrapes various pages for metadata, and then as necessary extracts videos by using the youtube-dl utility. The end result is a directory on my machine which always has a few interesting videos to watch, ad-free, tracking-free, network connection free, using my favorite full-featured video playback tools.

I think others seeking a minimalist youtube experience might enjoy youtube-dl: https://rg3.github.io/youtube-dl/

View on HN

You don't need to scrap the page, YouTube provides the RSS stream for each channel: `https://www.youtube.com/feeds/videos.xml?channel_id=[ID]`. The RSS contains the "normal" link of each video, so you can invoke youtube-dl or mpv on it.

> You don't need to scrap the page

They do if they are scraping playlists.

Youtube assumes that when creators make a playlist, they’ll add videos to the top. As such, the RSS for a playlist[1] always returns just the top 15 results. But a lot of creators — typically the ones that take greater care into organising their playlists, such as Crash Course[2] — add videos to the bottom of their playlists. Those RSS feeds will be stuck at the same 15 items forever, and your only option is to scrape the page.

[1]: https://www.youtube.com/feeds/videos.xml?playlist_id={playli...

[2]: https://www.youtube.com/user/crashcourse

You can use youtube-dl's archiving feature for that.

I use it regularly on my fairly large music playlist and it goes through all 20 pages just fine despite adding stuff on the bottom occasionally (I sorted it by creation date)

youtube-dl is really cool, and supports loads of sites as well as just youtube.

VLC can also play youtube URLs directly, although buffering can be annoying.

There's also a Python package called "whitey" which provides an ncurses interface for searching youtube, and sending videos to e.g. mplayer or youtube-dl.

If you want to "subscribe" without needing an account, every youtube channel provides an RSS feed (look in the <head> of the page source).

VLC can also play youtube URLs directly

I legitimately did not know this, thanks for the knowledge share! Can it follow playlist s of videos?

There's a few video game streamers whose commentary and remarks are so funny and well written I find myself listening to them more like a podcast than watching the video, so I'm often queuing up playlists and having the background noise while doing house chores and the like

VLC can't play YouTube playlists natively, but there's a VLC add-on that supports doing so: https://addons.videolan.org/content/show.php/+Youtube+playli...

Also, mpv can use youtube-dl if both are installed to play YouTube playlists, e.g.:

    mpv https://www.youtube.com/playlist?list=PLhyKYa0YJ_5DfJnFBZwz8xd37MA6I16oJ

Note that you need to pass it the link to the playlist itself and not a video in the playlist: https://github.com/mpv-player/mpv/wiki/FAQ#How_can_I_play_Yo...

Excellent, thanks!

Well, if you're on Mac and if you want to forgo the playback speed controls (like for, music) I can also recommend Vox earlier versions (2.8.26)[0]

It is an audio player with the youtube-list-as-playlist feature. In my experience, it generally delivers a better sound quality of the same videos than youtube. Copy the url (make sure it's /?list=<id>). Hit Cmd + U, paste, <Enter>. Then optionally download the extension for using keyboard buttons[1] and control playback from there.

Or you can always use `youtube-dl -ix /?list=` then open the resulting files with .opus extension with your favorite player.

[0]: https://vox.rocks/mac-music-player/old-versions [1]: https://vox.rocks/mac-music-player/control-extension-downloa... Disclaimers: No commercial interests. Just a happy user.

>youtube-dl is really cool, and supports loads of sites as well as just youtube.

Agreed, here's a list of sites it supports: https://github.com/rg3/youtube-dl/blob/master/docs/supported...

Not sure how much you'd benefit from it, given that you've already got a good setup, but I just made a website called HookRSSTube[0] that essentially proxies YouTube RSS feeds and replaces the video playback with HookTube[1] instead.

I'd love to know if you have any comments or suggestions.

[0] https://hookrsstube.com

[1] https://hooktube.com

+1 for hooktube! I now use a greasemonkey script to auto-redirect all youtube.com links to hooktube instead.

Do you have a way to ensure you get the highest quality?

I do similar with cron but often get videos as they're still processing (and thus only at lower qualities). One issue is the channels I 'subscribe' to don't always output at the same resolution and frame rate.. so I can't do a simple 'is video 4k60' test.

I don't think there's any way to know what qualities are currently being prepared. Your best bet might be to just figure out what time that creator typically uploads, and make sure your script doesn't run during that time and for several hours afterward, to give YouTube the time it needs to prepare the high-quality ones.

Could you look at the upload date and wait a certain amount until the processing is more likely to be done?

That's an excellent idea. You can work out a reasonable amount of time to allow YouTube to do the video processing, and then use youtube-dl's "--datebefore" option to only download videos that are at least that old. You'll have to manually compute the date and time for X hours in the past, I suppose.

Thanks for the hint wrt --datebefore. I'll give it a go :)

Ah! That could work indeed. Thanks :)

After it has finished processing, youtube-dl will (by default) download the best format anyway.

I actually have a script running on a cron job on my personal server that does approximately this. It downloads into a Syncthing folder that automatically syncs to whatever devices I might want to watch videos on. The difference is I just have a set of playlists for it to download, so I'm relying on the YouTube creators to organize their videos into appropriate playlists, which means it's not exactly "fine-grained". It's basically an 80% solution, and for the remaining 20%, I just invoke youtube-dl manually.

Uploads is a playlist.

For the few YouTube channels where I actually want to watch every single video uploaded by the creator, I do use this. For the rest, I have to hope that they create playlists for the subset of their videos that I am interested in. Many creators are actually quite diligent about organizing their uploads into playlists.

Could you release some part of that script too? It sounds really interesting.

If you like youtube-dl you should check out mps-youtube (https://github.com/mps-youtube/mps-youtube) which has the ability to use youtuble-dl as the backend I'm pretty sure (i prefer to use the native one personally)

It's also gpl vs youtube-dl's public domain.

I'm sure eventually I'll figure out a way to just do it in emacs. (I'm mostly joking... kinda... maybe.)

> mps-youtube

Funny that state[0] on PyPi[1] is [downloads|no longer available]

[0] https://img.shields.io/pypi/dm/mps-youtube.svg

[1] https://pypi.python.org/pypi/mps-youtube

PyPI no longer tracks download counts on its own. Instead, they pipe access logs to Google BigTable, so you can BigQuery the heck out of them.

> (i prefer to use the native one personally)

Then you should try - SMTube[0].

SMTube allow use "mpv + youtube-dl" for play videos.[1]

[0] http://www.smtube.org/

[1] https://blog.smplayer.info/how-to-download-videos-with-the-n...

I've been a youtube-dl fan for a long time -- it was essential whilst traveling a few years ago with suboptimal 3G bandwidth... these days I usually just paste the URL in to a terminal after typing `mpv` for a minimal experience but still with all the control you might want for playback speed, brightness, screenshots, & skipping around etc. Those who prefer a GUI could use SMplayer.

For anybody interested I've written a tutorial for youtube-dl: https://spapas.github.io/2018/03/06/easy-youtube-mp3-downloa...

I have a project that kinda acts as music assistant: "Search and play any song from terminal"

The project is somewhat inactive right now due to personal priorities. http://github.com/NISH1001/playx

How well does youtube-dl deal with video playback being restricted to play only from youtube.com? For instance this Tube site can't really play most music videos.

Well.

See also mps-youtube, a Python utility installable via pip.

> ad-free

You don't want the people who make the content you enjoy to earn a living doing that?

Reply on news.ycombinator.com