Youtube-dl | Hacker News Reader

Youtube-dl

rg3.github.io

597 pointspmoriarty169 comments

It's very nice to see a project I started reach the front page of HN.

I remember starting the project around 2006. Back then, I had a dial-up connection and it wasn't easy for me to watch a video I liked a second time. It took ages. There were Greasemonkey scripts for Firefox that weren't working when I tried them, so I decided to start a new project in Python, using the standard urllib2. I made it command line because I thought it was a better approach for batch downloads and I had no experience writing GUI applications (and I still don't have much).

The first version was a pretty simple script that read the webpages and extracted the video URL from them. No objects or functions, just the straight work. I adapted the code for a few other websites and started adding some more features, giving birth to metacafe-dl and other projects.

The raise in popularity came in 2008, when Joe Barr (RIP) wrote an article about it for Linux.com.[1] It suddenly became much more popular and people started to request more features and support for many more sites.

So in 2008 the program was rewritten from scratch with support multiple video sites in mind, using a simple design (with some defects that I regret, but hey it works anyway!) that more or less survives until now. Naturally, I didn't change the name of the program. It would lose the bit of popularity it had. I should have named it something else from the start, but I didn't expect it to be so popular. One of these days we're going to be sued for trademark infringement.

In 2011 I stepped down as the maintainer due to lack of time, and the project is since then maintained by the amazing youtube-dl team which I always take an opportunity to thank for their great work.[2] The way I did this is simply by giving push access to my repository in Github. It's the best thing I did for the project bar none. Philipp Hagemeister[3] has been the head of the maintainers since then, but the second contributor, for example, was Filippo Valsorda[4], of Heartbleed tester[5] fame and now working for Cloudflare.

[1] http://archive09.linux.com/articles/114161 [2] http://rg3.name/201408141628.html [3] https://github.com/phihag [4] https://github.com/filosottile [5] https://filippo.io/Heartbleed/

Another thanks here. I use youtube-dl so much that I occasionally substitute it for wget when trying to fetch online content (and occasionally discover in the process that it will in fact grab what it was that I was trying to get in the first place -- mostly audio files).

I vastly prefer offline media players to browser-based tools, for a number of reasons: better controls and playback, richer features, uniform features (I don't have to learn each individual site's idiosyncracies), the ability to queue up a set of media from numerous sources and play them back without clobbering one another, and more.

Hugely useful tool, and I've been impressed as hell as well by its update frequency.

And lift a mug to old Warthog. I miss Joe as well.

> I vastly prefer offline media players to browser-based tools

Why don't browsers provide some way to play local video files, for example by typing "file:///c:/my_video.flv" into the address bar. After all, the browser certainly includes the ability to play the video being downloaded off the web.

If you try "file:///c:/my_video.flv" with Firefox, it opens a dialog box offering to pass the video file to whatever external media players you have installed.

In what seems inconsistent to me, "file:///c:/my_notes.txt" and "file:///c:/my_pic.jpg" will be rendered correctly by Firefox -- it won't offer to open an external text editor or photo viewer. Why is video different?

Chromium browser let's you do this. It is how I stream my videos over chromecast. It also let's you browse your filesystem.

I tried Google Chrome (v29 on Windows, fwiw), and it behaves just like Firefox; i.e., it'll render local text or image files but won't play local videos. I didn't try Chromium.

You're right that FLVs don't work (it'll simply "re-download" the video). However, MP4s and WEBMs work fine (probably several other formats as well).

I'm using Chromium Version 38. It works for most videos but I have had some the audio did not work for.

Browsers don't natively handle .flv files, they hand off the file to the flash plugin. If you try that with files the browser can handle (.mp4 videos, .mp3 music, .png images, etc.) you'll see that it works fine.

Works perfectly for me in Firefox:

http://i.imgur.com/iCdmZhF.png

Filippo here. I'm sad that I didn't have the time to contribute to ytdl much recently, but it was my first time playing a role in a big and popular project, and I'm terribly grateful for that trust. (Thanks Ricardo, thanks Philipp!)

Also, what always impressed me is the incredible amount of random contributions from the community. Ever since we introduced a super-simple plugin system [0], support for the most disparate video sites poured in as PR. (>800 PR!!) Also, given how ytdl is structured, the most simple plugin gets you 90% of the tool power for that video sit. Big results with minimum effort.

Finally, to answer the question about the updates in some siblings, there is no active effort against us most of the time (VEVO videos being the notable exception) but supporting such a number of sites mainly by scraping means that breaking changes happen really really often.

[0] https://filippo.io/add-support-for-a-new-video-site-to-youtu...

Thank you for all your contributions! Can you update that article to use video_id = self._match_id(url) and _VALID_URL = 'https?://...' though? We've also added a fair bit of "official" documentation at https://github.com/rg3/youtube-dl/blob/master/README.md#addi... .

Love your tool. Thanks, like others say I use it at least a few times every week. I just prefer watching longer videos in VLC, easier seeking and no buffering, etc. Also much snappier than Youtube on my old laptop.

This won't solve the buffering problem, but for the record, you can use "open network location" in VLC to watch YouTube videos directly.

Regardless, youtube-dl and VLC complement each other quite well.

Great job rg3, thank you very much for your work!

Do you think DRM support in browsers and major tube sites will soon prevent tools like youtube-dl from functioning?

Well, this is going to be more of a philosophical answer than a technical one: with the popularity of YouTube nowadays, which is available on every platform and allows for anyone, anywhere to instantly watch a video, the cat-and-mouse DRM game would not succeed. I think DRM is flawed (insert the typical lock-and-key analogy here) but it does work for some situations. For YouTube: probably not. Somewhere, someone talented would crack it and tools like youtube-dl would continue to exist. A recent example is youtube-dl using rtmpdump when available to download DRMed videos.

I use this tool a couple of times every week. I love this tool!

And I must say I'm impressed by its ease of use (basically zero installation effort), and also by the frequent updates.

(I wonder why those frequent updates are necessary, though. Are you under the impression that google is actively working against tools which attempt to download material from youtube?)

Hi, I'm the current lead developer. We update extremely frequently because our release model is different from other software; there is usually little fear of regressions (fingers crossed), and lots of tiny features (i.e. small fixes or support for new sites) that are immediately useful for our users. We've had the experience that almost all users prefer it that way, so we try to enable every reporter to get the newest version by simply updating instead of having to check out the git repository.

As @fillipo said above, there is little if any pushback from video sites. Most of the time, they update their interface (we've gotten better in anticipating minor changes) and something breaks. The recent string of YouTube breaks (for some videos, mostly music videos - general video is unaffected) is caused by the complexity of their new player system, which forces us to behave more and more like a full-fledged webbrowser. But I think we usually manage to get out a fix and a new release within a couple of hours, so after a small youtube-dl -U (Caveats do apply[0]) you should be all set again. Sorry!

[0] https://yt-dl.org/update

I'm really grateful for this tool: both to the creator, and those that keep making sure it works. I have a swath of my life tied up in a couple of youtube playlists, and every now and then videos disappear (presumably due to DMCA-requests) -- and it's always annoying. With youtube-dl, I can simply download my lists, and then I can be confident that whatever obscure (or not) tune or cover version I found some years ago, will be in my archives (I've yet to automate this -- but my goal isn't really to archive all the things -- I generally add few songs at a time).

Anyway, if you didn't write this tool (and update it) -- I'd have to do it myself. And I'd rather not do anything myself ;-)

The current team should have more information, but I think most updates are due to other sites breaking and new sites being added that due to YouTube or bug fixes. I don't think YouTube is actively working against tools like youtube-dl, at all.

The frequent (almost daily, sometimes more frequent than daily) updates are really impressive.

I agree. It's a consequence of the software being very volatile, having thousands of users and supporting so many sites. There's something to fix or to add every day.

This tool ended up being so useful, it made the internet substantially better for me.

As the catalyst and original dev for this tool, Thank you!

youtube-dl lets me avoid the awful youtube ui.

I setup a makefile to let me just go make and then eventually vlc pops up with stuff to watch every so often. Its quite nice.

https://github.com/mitchty/youtubes/blob/master/Makefile

Thanks so much as well, I use FreeBSD and it has been the only way for me to watch youtube video consistently.

Remarkably, YouTube makes scripting downloads very easy. The script below needs only sed and some http client and it has worked for years. I have only had to change it once when there was a change at YouTube; the change was very small.

   # this script uses sh, sed, awk, tr and some http client
   # here, some http client = tnftp
   # awk and tr are optional
   
   
   # wrapper for tnftp to accept urls from stdin
   ftp1(){
   while read a;do 
   ftp ${@--4vdo-} "$a" 
   done;}
   
   
   # uniq
   awk1(){ awk '!($0 in a){a[$0];print}' ;}
   
   
   # some url decoding
   f1(){
   sed '
   s,%3D,=,g;
   s,%3A,:,g;
   s,%2F,/,g;
   s,%3F,?,g;
   s/^M      
   //g;
   #  ^ thats Ctrl-V then Ctrl-M in vi   
   ' 
   }
   
   # remove redundant itags
   f0(){
   sed -e '
   s/&itag=5//;t1
   s/&itag=1[78]//;t1
   s/&itag=22//;t1
   s/&itag=3[4-8]//;t1
   s/&itag=4[3-6]//;t1
   s/&itag=1[346][0-9]//;t1
   ' -e :1
   }
   
   # separate urls 
   f2(){
   sed '
   s,http,\
   &,g' 
   }
   
   # remove unneeded lines
   f3(){
   sed '
   #/^http%3A%2F.*c.youtube.com/!d;
   /^http%3A%2F.*googlevideo.com/!d;
   /crossdomain.xml/d;
   s/%25/%/g;
   s,sig=,\&signature=,;
   s,\\u0026,\&,g;
   /&author=.*/d;
   ' 
   }
   
   
   
   # separate cgi arguments for debugging
   f4(){
   sed '
   s,%26,\
   ,g;
   s,&,\
   ,g;
   ' 
   }
   
   # remove more unneeded lines
   f5(){
   sed '
   /./!d;
   /quality=/d;
   /type=/d;
   /fallback_host=/d;
   /url=/d;
   /^http:/!s/^/\&/
   /^[^h].*:/d;
   /^http:.*doubleclick.net/d;
   /itag.*,/d;
   '
   }
   
   # print urls 
   f6(){
   sed 's/^http:/\
   &/' | tr -d '\012' \
   |sed '
   s/http:/\
   &/g;
   ' 
   }
   
   f8(){
   sed 's/https:/http:/'
   }
   
   FTPUSERAGENT="like OSX"
   
   case $# in
   0) 
   echo|$0 -h 
    ;;
   [12345])
   case $1 in
   
   -h|--h)
   echo "url=http[s]://www.youtube.com/watch?v=..........."
   echo usage1: echo url\|$0 -F \(get itag-no\'s\)
   echo usage2: echo url\|$0 -g \(get download urls\)
   echo usage3: echo url\|$0 -fitag-no -4o video-file
   echo N.B. no space permitted after -f
   
    ;;
   -F)
   $0 -g \
   |tr '&' '\012' \
   |sed '
   /,/d;
   /itag=[0-9]/!d;
   s/itag=//;
   /^17$/s/$/ 3GP/;
   /^36$/s/$/ 3GP/;
   /^[56]$/s/$/ FLV/;
   /^3[45]$/s/$/ FLV/;
   /^18$/s/$/ MP4/;
   /^22$/s/$/ MP4/;
   /^3[78]$/s/$/ MP4/;
   /^8[2-5]$/s/$/ MP4/;
   s/.*?//;
   '|awk1
    ;;

   -g)
   while read a;do
   n=1
   while [ $n -le 10 ];do
   echo $a|f8|ftp1||
   echo $a|f8|ftp1 &&
   break
   n=$((n+1))
   done \
   |f2|f3|f1|f0|f4|f5|f6|f1|sed '/itag='"$2"'/!d'
   done
    ;;

   -f*)
   while read a;do
   n=1
   while [ $n -le 10 ];do
   echo $a|$0 -g ${1#-f} |ftp1 $2 $3 $4 $5 ||
   echo $a|$0 -g ${1#-f} |ftp1 $2 $3 $4 $5  && 
   break
   n=$((n+1))
   done
   done
    ;;

   esac
   esac

There are separate scripts for extracting www.youtube.com/watch?v=........... urls from web pages to feed to this script.

The problem is that this only works for some YouTube videos (for example it will fail for basically all VEVO videos), not to mention maintainability issues.

I had to look up what "VEVO" was. A joint venture of several major record labels and Google launched in 2009.

Personally I have no need for "VEVO" videos. Nor do I ever encounter VEVO youtube urls posted to websites, like HN. I wonder why?

As for maintainability, I beg to differ. The raison d'etre for this script arose out of frustration that early YouTube download solutions, e.g. gawk scripts, clive, etc., kept breaking whenever something at YouTube changed. I got tired of waiting for these programs to be fixed, if that ever happened.

I can fix this 164 line script faster if YouTube changes something than waiting for a third party to fix something they developed that is far more complex. Moreover, it does not rely on Python. Is there something wrong with DIY?

I see someone posted a link in this thread to another 208 line script, yget, that uses sed and awk. This further demonstrates the relative simplicity of downloading YouTube videos.

>Personally I do not watch "VEVO" videos but I am curious what they are. //

Go to youtube.com, you may need to scroll but most unlikely, bam! "VEVO" video with x-million views: it's a music video promotion brand.

Actually it's not globally promoted so outside of Western Europe and USA I'd guess you don't get VEVO vids so much?

According to https://www.youtube.com/watch?v=5zs1ClgqhLw their 100th most viewed video has 200 million views. Top 10 are all above 600 million.

They're quite a big brand.

Interesting.

An alternative to goofing around on the youtube.com web site, scrolling constantly and getting hit with advertising and endless lists of "related" videos is to search and retrieve youtube urls from the command line via gdata.youtube.com.

Pastebin next time, please.

Despite its name, youtube-dl doesn't just download from YouTube but from a ton a different sites as well [1]. The rate in which this project keeps up with changes is incredible.

[1]: https://github.com/rg3/youtube-dl/tree/master/youtube_dl/ext...

It seems like quite a modern success story for the classic "Cathedral and the Bazaar" model of open source development structure and motivations.

As I recall, it was originally written by one person (Ricardo Garcia) in 2008 and worked only on YouTube using (by later standards) relatively simple heuristics to find the URL to extract the video. But it's catalyzed an explosion of interest in every aspect of the problem: tracking changes to the HTML of the video sites, adding support for more video sites, figuring out indirection and parsing through multiple pages and HTML objects, making the tool much more multiplatform and easier to install and update...

It's attracted hundreds of contributors (many of them motivated by a personal desire to be able to use the tool on a different site, or to fix a bug that was preventing them from downloading video in a particular rare case) and maintained an incredibly rapid pace of development.

This is exactly why I contributed. In fact this morning, by coincidence, I had my first ever PR accepted, and it was for this piece of software [1]. I was using youtube-dl to download VK videos, but I really wanted to be able to download an entire playlist -- in the same way you can for YouTube. It didn't exist, so I just got stuck in and did it myself. It really helped that there was many other examples I could look at from other sites, and the maintainer of the package provided me with some very good feedback.

This kind of project that requires a lot of fairly laborious work to create support for many different information sources is a particularly good candidate for an open source project.

https://github.com/rg3/youtube-dl/pull/4233

I'm not sure if "leeching copyrighted content" was the kind of motivation that Eric Raymond had in mind for future open source projects when he wrote Cathedral and the Bazaar.

I call it "ability to watch Youtube content offline".

That was indeed the original purpose.

When I was in the Himalayas earlier this year, between poor WiFi and sketchy 3G, it was the only practical way to watch at all. Having the file offline was an added bonus that meant others could benefit too, so big thanks to rg3 & current devs on behalf of a lot of folk who've never been near HN themselves.

Yeah, because at the core of human civilization lies a respect for copyright, a BS notion that was developed for exploiting the restrictions of (analog) physical formats for profit...

Copyright law was developed in the 1700s precisely to prevent people from exploiting the limitations of physical formats for profit.

It's the opposite of what you've stated.

Authors, composers and publishers needed protection against cheap printing presses that would just print anything that was popular and flog it in the marketplaces.

>Authors, composers and publishers needed protection against cheap printing presses that would just print anything that was popular and flog it in the marketplaces.

The limitations I mention are the difficulties and cost of the printing itself.

What authors wanted was to restrict who can print their work -- but it's not true that authors "needed protection" because printing presses started appearing.

That makes it sound like authors were paid for the work until those "cheap printer" pirates appeared. But on the contrary it was the invention of the printing presses themselves that gave authors an industry in the first place -- for millenia authors just wrote for free.

> for millenia authors just wrote for free.

Yes; a good read is "The Surprising History of Copyright and The Promise of a Post-Copyright World" [1] which I think is from Karl Fogel, the author of the (Free, Libre, CC-BY-SA) book "Producing Open Source Software" [2]

[1] http://questioncopyright.org/promise [2] http://producingoss.com

The reason the industry of paid authoring could develop is because of copyright. Without it, all the value of the new printing industry would have accrued to the printers, and none to the authors.

Yeah, and culture would be free.

An anti-copyright freedom fighter downloading stuff from YouTube is not unlike an anti-capitalist punk rocker buying her clothes at H&M...

At least put some effort to make the analogy more accurate:

"it's not unlike an anti-capitalist punk rocker STEALING her clothes at H&M".

That said:

First, I fail to see the contradiction from being an "anti-copyright freedom fighter" and "downloading stuff from YouTube".

Someone somehow convinced you than anti-copyright people only like copyleft works? The very idea of being anti-copyright is wanting to abolish all copyright.

Second, what's witht the "anti-copyright freedom fighter" strawman? As if someone needs to be that to want to download stuff off of YouTube?

Like it or not, modern civilisation's engine is "profit".

Well, I don't like it, and I don't bend down for anything that's not a physical law just because "like it or not it's X".

Might as well have written "like it or not, the South's engine is slavery" in 1850...

I'm not sure "leeching copyrighted content" is a fair description of what Youtube-dl does. Yes most of the content you will download with it is copyright, but it is content that you already have a right to see, and in most cases the expectation is that you would maintain your right to see it for as long as Youtube (or other sight) remains active. The main difference is that Youtube-dl allows one to view the content with a program other then the browser. I suspect that few people uploading to a video sharing site did so with the intention of requiring people to view it using that sites player, but rather did so with the intention of people viewing it, and the player restriction was incidental.

The one place I can see where this breaks down is in advertisements, but I consider that to fall into the incidental results. (Although Youtube-dl does have a --include-ads option)

>it is content that you already have a right to see //

You have opportunity, that's not the same as a right. The content supplier is under no obligation to provide content to you, ergo no "right to see" that content.

That said, personal time-shifting and format-shifting should IMO be a normally allowed part of the copyright deal.

Including non-video sites like Soundcloud and it supports playlists (downloads multiple files) on and off Youtube.

I love how many porn sites are supported by this

There's not much motivation to create a non-porn site YouTube clone. You have to believe you can do it better or need to not host your content on YouTube, and you have to be able to do it.

Porn has the need (the mainstream providers generally delete porn) and the sheer resources to do it.

If there is a will, there is a way.

The playlist feature is actually one that typically annoys me. So I've just re-read the manpage and found the --no-playlist argument.

Sorry! The problem is that our userbase is split about wanting the playlist or the video. You can create a file ~/.config/youtube-dl.conf with the content --no-playlist so that you don't have to type it out every time.

Right. My point was that the mention prompted to me to read TFM and find the fix.

I've just done the config file setting you mentioned.

I've installed youtube-dl via homebrew. How do I install these external sources?

You don't need to, as far as I'm aware. I think you can just request a supported URL, and it will download it automagically.

Correct. Each class that handles information extraction for a different side defines a regexp to match the url against. (Note: Some of the regexps aren't hardy to the http-vs-https distinction, so you might have to remove the 's')

Awesome, thanks. Should have tried before asking. :) Searched my system drive for these .py files, but found nothing, so I figured something was missing.

All of the modules are compiled into a single file for the youtube-dl command. I've never looked into what they are using to do this, but you could poke your head into the repo to check it out.

We're simply making use of Python's ability to load a module from a zip file [0]. Therefore, the generation[1] is just zipping up all the files and prepending a shebang.

[0] http://bugs.python.org/issue1739468 [1] https://github.com/rg3/youtube-dl/blob/640743233389714dda8a3...

This:

  youtube-dl: youtube_dl/*.py youtube_dl/*/*.py
  	zip --quiet youtube-dl youtube_dl/*.py youtube_dl/*/*.py
  	zip --quiet --junk-paths youtube-dl youtube_dl/__main__.py
  	echo '#!$(PYTHON)' > youtube-dl
  	cat youtube-dl.zip >> youtube-dl
  	rm youtube-dl.zip
  	chmod a+x youtube-dl

Might be less confusing if you append '.zip' in the first two commands:

  	zip --quiet youtube-dl.zip youtube_dl/*.py youtube_dl/*/*.py
  	zip --quiet --junk-paths youtube-dl.zip youtube_dl/__main__.py

When you echo the shebang overwriting the file, I was thrown off. I'm thinking, "Why did you just zip all those contents into the file to just throw them out?" Then I see the `cat` line, and it makes sense that the `zip` command appends the .zip to the end of the file.

YT videos lag a lot when I stream them directly in VLC 2.2.0, (`vlc <youtube link>`) and some protected videos don't play at all, so I often use this:

    ytplay() { youtube-dl "$1" -o - | vlc - }

As a side benefit it of course also allows you to instantly watch stuff from all the other sites YT-DL supports :)

That's a great idea. I use Livestreamer for that purpose.

https://pypi.python.org/pypi/livestreamer/1.10.2

Had to add a colon:

    ytplay() { youtube-dl "$1" -o - | vlc -; }

but works great!

That is a semicolon, and it shouldn't be necessary (at least not in bash/zsh).

I believe it's necessary, otherwise the closing bracket counts as parameter to the last command.

Correct. On the command line:

    ytplay() { youtube-dl "$1" -o - | vlc - ; }

or:

    ytplay() { youtube-dl "$1" -o - | vlc - #<enter here>
    > } #Where "> "  is bash prompting for more/end of definitoin

In a file (eg: .bashrc), I'd personally prefer:

    ytplay() {
       youtube-dl "${1}" -o - | vlc -
    }

Note that there's very little difference between "$1" and "${1}" in practice, I tend to prefer it for consistency with recommended[1] practice of using ${NAME} rather than $NAME. (And to differentiate something like "${1}${2}" vs "${12}", as you might if $1 was a name, and $2 an extension, or $1 and url-scheme and $2 a host-name (http://hostname -> "${1}${2}" 1="http://" 2="hostname").

[1] http://stackoverflow.com/questions/8748831/bash-why-do-we-ne...

May I ask what the program "ytplay" is? Google doesn't guide me.

It's a function name that renekooi would have made up to run the command following it without having to type it all out every time.

That's how you define a function in Bash.

Holy heck that's a great idea. Thanks!

I use xargs

    youtube-dl --get-url "$1" | xargs mplayer

I think I fixed the "some protected videos" last week.

If not, please share me a YT link.

Quick protip for those wondering, the simple command to download an entire youtube channel is like so:

$ youtube-dl -citw ytuser:LastWeekTonight

I downloaded a channel with 121 videos, 4.4 gigs, took 26 minutes, so 2.8MB/s average. Curious if the Youtube people will shrug it off and free the beer or rate limit or more aggressively combat this.

Also, to get the total number of supported sites:

$ youtube-dl --extractor-descriptions|wc -l

466 (wow)

As this can run on anything with Python, I guess that includes Android[0], iOS[1], Windows Phone[2], heck even Blackberry[3]??

[0] https://python-for-android.readthedocs.org/en/latest/

[1] https://code.msdn.microsoft.com/windowsapps/using-python-on-...

[2] http://pythonforios.com/

[3] http://forums.crackberry.com/blackberry-z10-f254/blackberry-...

Thanks pmoriarty for submitting this. Awesome and I'm just getting started poking around with it. Makes me really want to learn Python, seems that's what all the fun stuff[4] is coded in.

[4] http://motoma.io/pyloris/ :)

Please don't pass in -citw [0]! I have personally run youtube-dl on android, works fine. (Disclaimer though: I am the current lead developer, so may have missed a pitfall or two).

[0] https://github.com/rg3/youtube-dl/blob/master/README.md#do-i...

Downloading an entire channel is exactly how I downloaded the How to Start a Startup class from youtube. For the interested:

  $ youtube-dl -citw ytuser:UCxIJaCMEptJjxmmQgGFsnCg

fredoliveira: Eyeballing those lectures, which are enormous, youtube-dl is helping you more easily teach people to make their lives and careers better, instead of just downloading Taylor Swift songs. So youtube-dl is indeed being used for Good.

Instead of Python For iOS, try Pythonista[0]. It's amazing.

[0]: http://omz-software.com/pythonista/

This may sound surprising but via youtube-dl I bought more music than before. If I find a new band that I might like, I search for Youtube videos first. The non-official videos often show just the cover of the CD or some useless slide show, so I extract the audio to have it in my playlist. Once I am decided that I like the music I had over to Bandcamp or Amazon to buy the mp3s. As an example: I lately bought four digital cds from progmetal act Redemption because someone upped their cd 'This Mortal Coil' to Youtube.

A very nice utility to have installed.

It can also convert a video to an mp3:

youtube-dl --extract-audio --audio-format mp3 https://www.youtube.com/watch?v=OKbtC223e30

  > youtube-dl -F OKbtC223e30
  139         m4a       audio only  DASH audio   49k , audio@ 48k (22050Hz), 525.88KiB (worst)
  171         webm      audio only  DASH audio  129k , audio@128k (44100Hz), 1.27MiB
  140         m4a       audio only  DASH audio  129k , audio@128k (44100Hz), 1.36MiB
  172         webm      audio only  DASH audio  176k , audio@256k (44100Hz), 1.78MiB
  141         m4a       audio only  DASH audio  255k , audio@256k (44100Hz), 2.71MiB

Just use youtube-dl -f 141 to direct download just the audio from YouTube. I always use this to download songs.

While -f 141 is of course perfectly fine, may I suggest -f bestaudio ? That should work fine for non-YouTube sites (soundcloud or so), and will get you a better version should YouTube one day add it. If you really prefer 141, you can also use -f 141/bestaudio to fall back to bestaudio if 141 is unavailable.

That'll come in handy. I always used -F to find out what is highest # for downloading audio.

Converting it to MP3 is a bad idea. Youtube uses other lossy codecs inside, such as Vorbis and AAC. So reencoding it will degrade the quality. The best option is to keep the audio as is.

According to the docs it formats them as "best" by default.

    --audio-quality QUALITY

    "best", "aac", "vorbis", "mp3", "m4a", "opus", or "wav"; "best" by default

When you convert audio from a lossy format to a lossy format (from YouTube's native AAC streams to MP3,) you always end up with worse quality than the original, regardless of the encoding settings. Since pretty much everything can play AAC, there's no point in converting it in the first place. Just remove --audio-format mp3, and you'll get an .m4a straight from YouTube with no conversion step.

That's right, usually the best quality audio you can get from youtube is in m4a format. The only problem I'm having is youtube sets the format inside the m4a to dash, which some stupid players (including iTunes) don't want to play. So I have to run something like

ffmpeg.exe -i "Keith Wiley - The Fermi Paradox, Self-Replicating Probes, Interstellar Transport Bandwidth-AUk6ZlePtQA.m4a" -c:a copy 2.m4a

to change the format in the file header to M4A.

Winamp and VLC also don't play these..

By the way, is DASH a free technology, or it's patent encumbered like the rest of the MPEG stuff?

Hmmm, can you add the --audio-format mp3 to ~/.config/youtube-dl.conf without breaking normal video downloads?

(Edit) Answering my own question, apparently yes!

Or even shorter:

youtube-dl -x --audio-format mp3 OKbtC223e30

In general youtube-dl is great, but that short id input handling is broken, as evidenced by this:

  youtube-dl -x --audio-format mp3 -fCtvurGDD8

As others have pointed out, you'll need -- in this case. However, there's really no reason why youtube-dl should not detect this common problem (we also try to detect when users forget to quote URLs with ampersands). Update to youtube-dl 2014.11.23.1 or newer and try this again ;)

By the way, the GitHub issue tracker (https://yt-dl.org/bug ) is usually a better place to report issues. But just for youtube-dl reaching #1 on HN, I'll make an exception.

Like many tools, youtube-dl supports a -- flag to indicate "the remaining arguments are not flags". So your example would work as:

    youtube-dl -x --audio-format mp3 -- -fCtvurGDD8

Because it interprets video IDs that start with a dash as short options? This should be better:

  youtube-dl -x --audio-format mp3 -- -fCtvurGDD8

This is the best video downloader. It has downloaded videos from every site I've thrown at it. It even downloaded videos from Comedy Central!

Edit: https://rg3.github.io/youtube-dl/supportedsites.html

I use this all the time and It's really great.

I have messed around with one sketchy youtube downloader or another for years until I found this.

Oh, and it's on brew!

I've used this quite extensively. It's less critical for YouTube now that almost all YouTube videos work with the HTML5 player, but it helped quite a bit when every other video required Flash. Still necessary for many third-party sites as well.

It'll also download an entire playlist, and add sequential numbers at the beginning (with the -A option).

I've also used it for downloading all possible formats for a video when debugging problems that only seem to be there in some resolutions.

Yeah, the format enumeration works quite well too. I've used it to download the "original" format for videos available in higher-than-1080p resolutions, as well as using the --extract-audio option.

No one pointed out that it has a do-whatever-you-want license. This is the thing which bothers me most. There are gazillions of shitty youtube downloaders (paid, free and adware supported) out there that people still use and the code is being powered by the work of open source developers.

I don't think the license would change anything, even if they chose something like AGPL. The sites would either ignore it (hard to prove infringement) or put a small link with their source. They don't rely on being proprietary anyway.

Well it'll atleast do something to limit creations and who knows if the copyright attribution was placed on many projects, youtube-dl would have received more attention, I'm surprised it wasn't already posted to HN.

It's very nice though for legitimate developers.

I think it's a legitimate choice. Personally I'm not worried my work is being used that way, as long as the original project is available and works.

As a stalwart defender of the WTFPL in my small circle of developer friends (who mostly think I'm crazy for it), you are now added to the list of people I can use to back up the fact that I'm not. ;)

Seriously though, awesome project, used it for a while.

Another example for you: libcaca. [1] However, the GNU people recommend the X11 license for small programs and Apache 2.0 if you've already decided to use a permissive license on a large program [2]

[1] http://caca.zoy.org/wiki/libcaca [2] https://www.gnu.org/licenses/license-list.html#WTFPL

I use this to replace noisy audio on a smartphone recording of dancing lessons with a high-quality version from a youtube video, automatically: http://youtu.be/AVIHpaNQLS0

Wow that's quite an impressive tool!

I mean I understand that all the separate steps are stuff we've seen is easily possible nowadays, but putting it together in a single UI makes it (roughly) 3000x as useful!

I want to try it, anyone had luck compiling it for Linux?

EDIT/update: Well I gave it a try, grabbed QtCreator, loaded the project, not much luck. Some issues with the "phonon" library, it seems. I'm not very good with getting C++ stuff to work when it gives build errors. I did spend about half an hour fiddling and googling error messages, but now it's time to give up, sorry :)

I'm writing this update to let you know that one of the errors I did manage to fix, is that Windows has case-insensitive paths/filenames, while Linux does not. Apparently the path for the phonon library is lowercase, so you should `#include <phonon>` lowercase. I'll try to leave a Github issue about this.

That didn't help much (complaints about the State enum in soundfix.h) which I tried to fix by also putting `#include <phonon>` in the soundfix.h. I'm not sure if that was right at all, but it did seem to fix that particular problem. As a result I was greeted with a whole bunch of other (I think unrelated?) errors about some types not being strictly compatible or something. That is where I check out until I know more about C++, decided it had been long enough, and just writing you a little message to let you know how it went.

However, it made me install and try QtCreator, something that I was meaning to do anyway. So that's a win :)

I'll try compiling it on qt5/linux as well, thanks for tracking down the phonon issues!

There's also a GUI wrapper for it: https://github.com/MrS0m30n3/youtube-dl-gui/

And a php web app that uses youtube-dl as a a backend: https://github.com/Rudloff/alltube/

Youtube-dl is the biggest pre-built thing I use in GifMachine[0] after ffmpeg, and I've used it in innumerable projects since then. I love youtube-dl, it's fantastic!

[0] - http://gifmachine.xwl.me/

Nice gifmachine - Thank you!

Is there a plugin version of this program, which would dynamically change any (supported) flash video reference to an HTML5 video tag? That way I can get rid of flash completely.

Don’t know about this program, but I use this on OS X Safari:

http://www.verticalforest.com/youtube5-extension/

Helped to get rid of Flash, although I have Chrome in my programs folder for sites that are still flash only.

I've setup this function in my bashrc to check for the latest build if the new one is more than 24 hours old and then run it with arguments supplied:

function youtube-dl {

        exe=$HOME/bin/youtube-dl
        link=https://rg3.github.io/youtube-dl/download.html
        url=$(curl -s $link |grep ">sig<" |head -1|sed -e 's/href="/|/g' -e 's/">/|/g'|cut -d"|" -f2)

        fetch=N
        if [ -s "$exe" ] ; then
                ts=$(date "+%s")
                yts=$(stat -c "%Y" $exe)
                [ $(( ($ts-$yts)/(60*60) )) -gt 24 ] && fetch=Y
        else
                fetch=Y
        fi

        if [ "$fetch" = "Y" ] ; then
                url=$(curl -s $url |grep ">sig<" |head -1|sed -e 's/href="/|/g' -e 's/">/|/g'|cut -d"|" -f2)
                echo "Fetching [$url] and deploying to [$exe]"
                curl -s $url -o $exe
                chmod a+x $exe
        fi
        [ -z "$@" ] && $exe --help || $exe $@
}

sounds like a job for a package manager ;-)

youtube-dl gets updated very frequently and the version that comes with your distribution (ie, ubuntu) usually is out of date and doesn't work often on most sites. So its better to download from the source and update it using "youtube-dl -U"

That's true for the version in Debian stable as well. The package in unstable tends to pick up upstream updates pretty quickly, though, especially when they're needed to fix site support.

The amount of time it takes to keep up with all of the changes big sites make is impressive.

At some point I decided to write something similar in Ruby ( https://github.com/rb2k/viddl-rb ) and I'm kind of ashamed of how broken things are from time to time.

Video hosting sites don't have APIs and reverse engineering the sources for the videos is like shooting at a moving target.

So kudos for leading that project :)

OK. I just realized this does something really cool. I've been troubled with 1080p videos as they no longer contain audio. They're separated and YT uses DASH to join the audio+video stream.

youtube-dl seems to solve this: https://github.com/rg3/youtube-dl/issues/2165

I just use quvi for that.

Example:

    quvi -vm --format $format "$url" --exec 'wget %u -O %t.%e'

instead of $format put any format that the video supports. To query them, use:

    quvi --query-formats "$url"

So it's going to be something like:

    quvi -vm --format fmt43_360p "$url" --exec 'wget %u -O %t.%e'

And to extract audio from the result you can use:

    avconv -i something.webm -vn -acodec copy something.ogg

Youtube however is switching away from fixed video files to separate streams to be used with MSE. You can note that higher resolution video is not available the old way. So downloading that won't be so straightforward.

youtube-dl supports muxing the separate video and audio streams from YouTube. You just need a recent ffmpeg/avconv and youtube-dl -f bestvideo+bestaudio.

This is way cool as YouTube separates video and audio streams for 1080p (and up) and then joins them with DASH.

Thanks, good to know. I'll try it out.

What actually happened to quvi? Is it really still active? (https://github.com/legatvs?tab=repositories). No more commit since last year.

Same question for clive/cclive...

By the way, FFmpeg has support for the 0.4 branch of libquvi, so if you built it with --enable-libquvi you can ffmpeg -i http://youtube... (assuming libquvi and its scripts still work)

> What actually happened to quvi?

Good question, I didn't keep track of its development. Your best option would be asking developers if they are planning to work on it further or not.

> By the way, FFmpeg has support for the 0.4 branch of libquvi

Same as mpv I think. You can play Youtube videos with it directly:

    mpv "$url"

Which is kind of fun, since you can do tons of things that aren't available in the browser player - looping, playing only portions of the video and all other things which mpv can do.

Note that quvi support has been removed on mpv git recently. They seem to be considering a switch to youtube-dl instead. https://github.com/mpv-player/mpv/commit/9b45b48c466765b6491...

That's interesting. How are they planning to use youtube-dl? Does it have a C library or mpv allows some plugins in other languages?

The bit that invokes youtube-dl is a Lua plugin. It checks if the URL starts with http:// or https://, and if so, invokes youtube-dl -J on the command line to dump JSON information about the video.

Awesome! Not long ago I switched to mpv from mplayer. mpv being able to use youtube-dl without any glue script would be fantastic.

Ah, I see. Thanks for clarifying.

I think quvi might be dead. mpv-git actually dropped libquvi a little while back because it was buggy and development was inactive. It's been replaced with a youtube-dl based downloader, so in mpv-git, this would invoke youtube-dl to get the video:

  mpv --ytdl "$url"

I wonder if there is any way to specify to it which stream to pick, since Youtube has many options. By default it picks H.264 / AAC, at least when it's using quvi.

IIRC --ytdl-format is passed directly to youtube-dl -f, but all other options have to be set in youtube-dl's config file.

If you don't want to install anything, just use http://savedeo.com for almost any video or http://auderio.com for Music from Youtube

Thanks, been using listentoyoutube.com but auderio.com is definitely better (less annoying popup stuff + required clicks)

And way faster :)

How does it compare to get_flash_videos, which is available in Ubuntu repository? https://code.google.com/p/get-flash-videos/

get-flash-videos supports Hulu; I've been meaning to try it out [1] because flash is Satan's anus. But I'd still keep youtube-dl for downloading & converting YouTube videos.

[1] https://gitorious.org/get-flash-videos-plugins/pages/Hulu

Is there a way of bypassing geoblocking via youtube-dl directly?

Well, you can get a cheap VPS in the US, or UK, or wherever the content is available, and run youtube-dl there. You can also use a proxy[1]. Unfortunately, I think socks proxies are not supported yet[1].

[1] https://rg3.github.io/youtube-dl/documentation.html#d5

[2] https://github.com/rg3/youtube-dl/issues/402

There are lots of projects to do this, and this one has been around for quite some time. Is there any context as to why to post this now?

If it's on the front page, some people have upvoted it.

Take a look at the "new" section!

I didn't know about this so I'm glad it popped up on the front page.

Someone is looking for easy karma points probably :)

Till now I didn't know that other Sites are supported :D wow really impressive site support! Thx for maintaining this nice tool :)

I've used this one for so many things and it's awesome.

I've downloaded a whole youtube channel, downloaded videos as mp3's and what not.

I have couple of aliases set in my .zshrc too :)

mp3dl() { youtube-dl --extract-audio --audio-format mp3 $1 }

root@haseebr7 ~ mp3dl <youtube_video_url>

and bam i have the mp3 download. I no longer have to visit shitty ad infested websites to do these kind of things.

I do the same, but have ethe -K option also whch keeps the original DLd video.

It seems like this functionality should be part of YouTube itself, at least as an option settable by the uploader.

youtube-dl rocks. One of my favourite features: youtube-dl -F

Yget is an alternative I've found to be more reliable. It's just for youtube though, and doesn't support all the things (such as bypassing age restrictions).

http://repo.hu/projects/yget/

How is it more reliable?

Back when I used youtube-dl, it seemed like Google changed something about Youtube every couple months and youtube-dl would break. Yget survived many more of these changes.

Getting youtube-dl to run has been a pain for some reason, for example I always seem not to have the right version of Python available. Yget doesn't depend on such volatile tools.

This appears to be the only option for Free Software access to soundcloud.com.

It's easy to just throw the streams of Soundcloud songs against whatever music player you have at hand. MPD even has a playlist plugin, which correctly handles basic soundcloud.com urls and handles all the API stuff for you.

On OS X, you can use afplay(/usr/bin/afplay) to play those downloaded videos' music(in a headless player). This is pretty useful if you listen to youtube music at work.

If I can lazyweb on this for a moment, is there a good script out there that can use this to sync/download your entire watch later queue?

youtube-dl can do that out of the box, try

youtube-dl --username pimlottc :ytwatchlater

If you get a problem, please file a bug report at https://yt-dl.org/bug . Thanks!

I really like this tool and it has been very reliable for me so far. Thanks for sharing the code.

I'm just using https://github.com/np1/mps-youtube

Oh man, I had that ages ago and used it frequently! It stopped working, if this is working again that's awesome :-)

You should update it often, as internal formats, urls, options etc of the sites change.

What's so special about this?

Probably lots of hackers want to be freed from the bondage of youtube.com while still retaining access to the wealth of content available there.

? hence the "so special" in my question. There are plenty of these tools already and it's been so for long time. As a hacker you should know. So again, what's so special about this?

It would be nice if it supported Netflix. There's some difficulties with this, but a discussion about it here: https://github.com/rg3/youtube-dl/issues/1564

I just use chrome extension

Reply on news.ycombinator.com