How journalists use youtube-dl
freedom.press
freedom.press
Speaking with them, it seems the associated metadata (json/xml?) and output log contain valuable information to go along with the video from an evidentiary perspective. Also the fact that youtube-dl has many test cases in the code goes towards demonstrating its reliability as an instrument for collecting evidence.
Certainly seems a better approach than installing a 'DownloadVideosEzy' extension for chrome or similar.
I think we can tell what the RIAA thinks about that.
Being public doesn't mean publicly owned. It means publicly accessible... Like your local shopping mall.
If we have to play by those rules, LEO should too. Boo hoo if it's harder.
They haven't shown anything. They made a claim. We'll know what rules we have to play by if it comes to a court ruling.
If I have a subject who has relevant video on a service like YouTube, and I decide to go the subpoena route, here's the process:
- Find the exact video, document the URL, Content creator's account name, and video upload date.
- Submit a preservation letter to Google to ensure the video will still be accessible when the subpoena is auctioned.
- go to my prosecutor/legal team to determine what jurisdiction the subpoena has to go through. It could be the county I'm in, the county the subject lives in, the county the video was uploaded in (unlikely), or the county where Google hosts the video.
- draft the subpoena and get it reviewed by whichever judge/magistrate applies.
- wait who knows how long for the subpoena response, hopefully it comes back with the real video and important content.
- face defense arguments in court that my subpoena was improperly obtained/submitted
No, it doesn't. And that "forever" are human rights that are established and should require a subpoena to contest.
(I can imagine lots of cases how this could be used to fight police corruption.)
If you post a video of murdering someone publicly - no subpoena should be required.
* youtube-dl is not just for Youtube (ie. google). It supports heaps of sites, and youtube is probably less likely to have the sort of content I'm referring to than twitter, pornhub, liveleak, who knows.
* I am in Australia, so for a subpoena (or similar) my friends would need to issue an MLAT request, which would indeed take a very long time. In fact, this often does happen, but in the meantime a formal preservation request (to the service provider) and a local copy (via something like youtube-dl) are important steps.
* Many investigations don't go to court, for a litany of reasons (that's a pun I guess). In such cases, a subpoena isn't in context: law enforcement have to investigate what has happened, to figure out if it needs to go to court.
* Some investigations are important but do not meet the 'serious crime' threshold for forcing companies or parties to provide information. If you can imagine being the victim of a minor crime, that someone filmed and put on twitter, I'm sure you can imagine the local police may want a copy of the video before the tweet gets deleted - but also know that there will never be an international cooperative justice process. It's just a local issue.
* Publically posted video is very common and often important, and it is not surprising that the police need a tool to download it. There are so many sites and web browser technologies, and the cops are not efficient spending their time coding youtube-dl equivalents.
I feel quite strongly about youtube-dl being taken down for copyright infringement by the RIAA, but have tried to answer in an informative way. Sorry if it comes across harsh - no ill will intended :)
Suppose there is some legitimate non-infringing use of some material. Interoperability, law enforcement use, fair use, accessing public domain material which is distributed using the same technological measures, whatever.
You create and distribute a piece of software to interface with the material for those purposes.
It's the same piece of software as one designed for any other purposes, because the software has no knowledge of your intent and your intent has no technical effect on its operation.
If that is allowed, doesn't that make 1201 a dead letter? If you make the tool available for non-infringing uses, the tool is available.
But if that isn't allowed, doesn't that make 1201 a contemptible offense against all of those important interests? And possibly unconstitutional as a result?
A tool designed to be used by police and marketed towards police is legal, as its intended purpose is not infringement. A tool advertised as "hey, kids, use this to infringe those copyrights" is totally different, even if it's literally the same tool.
Moreover, that would save the law from being a dead letter in the sense that there would be circumstances when it could theoretically be enforced (i.e. when someone is overtly marketing it as a tool for infringement), but wouldn't anybody making the tool then just not do that? It's still exactly the same tool. And then what does the tool have to do with anything anyway, when what you really have is a law against speech promoting copyright infringement?
HN is not a court of law, most people here are not lawyers, let alone well-paid and competent lawyers like the ones the RIAA can afford.
> wouldn't anybody making the tool then just not do that?
Sure - which is why you're not seeing many ads for tools to steal cars or pick locks. But it's not only about the way the material is promoted, it's also about how it's obtained, whether the seller could reasonably know what it would be used for, and a bunch of other stuff.
You can't look at this from a binary perspective; despite depictions in popular media, the law is typically not about "aha" moments, but rather about putting a number of coherent pieces together to paint a certain picture.
---
As a matter of policy (as well as legality), youtube-dl does not include support for services that specialize in infringing copyright. As a rule of thumb, if you cannot easily find a video that the service is quite obviously allowed to distribute (i.e. that has been uploaded by the creator, the creator's distributor, or is published under a free license), the service is probably unfit for inclusion to youtube-dl.
A note on the service that they don't host the infringing content, but just link to those who do, is evidence that the service should not be included into youtube-dl. The same goes for any DMCA note when the whole front page of the service is filled with videos they are not allowed to distribute. A "fair use" note is equally unconvincing if the service shows copyright-protected videos in full without authorization.
Support requests for services that do purchase the rights to distribute their content are perfectly fine though. If in doubt, you can simply include a source that mentions the legitimate purchase of content.
---
Now, if what you really mean is that is is vague, then I agree. But I don't see any case for it being incoherent.
Except that it isn't, as already discussed, because the code doesn't change based on the design intent. The technical operation of the tool is identical whether it's used for interoperability or fair use or to rip songs for The Pirate Bay. The technical operation of the tool is identical whether it's produced by a university researcher or the developer of a media player app or a literal sea-faring pirate with a parrot and a peg leg who is the founder of a political movement to abolish copyright and promote civil disobedience.
But prohibiting a specific person (who may have had a particular intent) from distributing it is incoherent (or at least futile) if it means you can just get some other person with some other intent to distribute the exact same thing. And it's also incoherent to say that the other person can't distribute it even if they genuinely do have a different intent, if the intent is supposed to be what matters.
None of which depends on what you judge a give person's intent to be after the fact, because it's a defect in the legislation, not the judgment in any particular case.
It requires that the piece of software is meant primarily for circumventing technical measures in place to prevent infringement. Just infringing copyright isn't a violation of the section.
https://news.ycombinator.com/newsguidelines.html.
(I understand the environment variable convention but if you think about the reason for the above rule, it's clear what needs to take precedence.)
Aside: Wonder how gitlab handles DMCA requests.
https://about.gitlab.com/handbook/dmca/
For a self-hosted instance, obviously it would be different. In that case, I assume the notice would go to the webhost.
Yt-dL is surely nicer and more convenient, but from an evidence standpoint, a one time $500 (to go wild) setup should have you covered, no?
(Mourning the loss of ytdl myself but trying to be realistic)
I am not saying it’s as convenient (more options >> less options except for analysis-paralysis). But I don’t understand how it tips the scale to making any archiving or evidence gathering unusable or uneconomical. (I am not saying GP is wrong - just want more explanation so I can understand)
Various detectors are more or less thwarted by it. It actually surprised me how strong the artifacts from some GANs are - they can survive several passes of re-encoding, but accuracy does suffer.
But I still need the raw 720/1080 stream for training.
* the quality drop; recompressing, non-matching frame rates, non-matching resolution--all the same goes for audio. You're very likely clipping (losing) data. This is assuming you're doing screen-capture. If you're literally video taping a monitor you will get moiré in the video, room tone in the audio, losing any stereo separation, and other audio/video artifacts.
* performance; must be done in real-time, cannot queue up multiple sources. This is likely the biggest efficiency killer and makes things 100x more labor intensive.
* reliable Internet; if you get a blip or have a slow connection you have to hopefully catch it and start over. With youtube-dl you can pause, resume, confirm even on the slowest, spottiest connections.
* metadata, organizing, indexing; likely hand-typed separately, prone to error, prone to not knowing if you've done that video already.
* Chain of custody; grabbing the original video allows you to prove two identical copies match (using file hashes or other comparisons) screen recording makes that difficult to impossible to confirm--maybe with fancy AI you'd have to run by the courts?
> the quality drop; recompressing, non-matching frame rates, non-matching resolution--all the same goes for audio.
Are you trying to preserve quality or prove something? My response was in context for "gathering evidence" but not police work, and not archival quality. Would such a copy cause your problem to prove libel, copyright infringement, illegitimate disclosure, etc?
> performance; must be done in real-time, cannot queue up multiple sources
Can most definitely queue up multiple sources. Just make a youtube playlist and record it. Yes, it takes "real time latency", you'll take 10 hours to download 10 hours of video in general -- that's not an issue for evidence or gathering in a non law-enforcement context.
> metadata, organizing, indexing; likely hand-typed separately, prone to error, prone to not knowing if you've done that video already.
Again - consider the context of my answer, NOT archival quality anything. The "cc" stream GP mentioned, which can be searchable etc - has also seen many revisions for many files when the Google STT algorithms are revised, and with corrections.
> Chain of custody; grabbing the original video allows you to prove two identical copies match (using file hashes or other comparisons) screen recording makes that difficult to impossible to confirm--maybe with fancy AI you'd have to run by the courts?
You have no chain of custody. You can prove two downloads are the same, but YouTube does not guarantee they keep the file the same (indeed, they've modified files several times, changing formats and even remastering old '80s videos). If a file is later pulled (which is what GP was talking about), what are you going to compare it to?
Chain of custody is law enforcement business. They'll get the files from YouTube directly, with affidavits and statements about it and any modifications, if they need it in court. You are going to civil court, and youtube-dl is not making your evidence more valid than a screen recording.
Did I expect anyone, ever, to propose that? Let's just say, 2020 is full of surprises. I want off this wild ride.
But when I find myself on a locked down computer - e.g. watching a movie on an AppleTV, or when I was shown surveillance video but was refused a copy for some bureaucratic reason (or it required a different license to export, reasons were unconvincing) - I use a mobile device.
Filming the screen, means that in order to fake it, you have to setup something that routes youtube.com to your own fake version of youtube, before filming. To me, that sounds much harder than say "this file was downloaded from here on that date"
I kind of expect a serious investigator to archive these materials just for the sake of it. I don't expect them to make it harder on themselves for no good reason.
It's just a GitHub repository which is lost, not ytdl itself.
Imagine if source code was posted on some type of blockchain like ETH2.. Or a decentralized file sharing service, like IPFS. I wonder if websites or self hosted interpreters using some kind of services like those would make serving a take down request nearly impossible.
Well for now we can pay m$ to host code in ways people rely on for the long term. Until, they don't and the codes all taken down in something analogous to modern book burning.
https://sfconservancy.org/blog/2020/oct/26/microsoft-github-...
I find GitLab far superior in every regard: its UI, CI stack, Auto DevOps, K8s integrations, on-prem/self-hosting option, support options, pricing structure, customer service, communications and open nature, frequent updates with great release notes, and more.
MS has two hats in the game here: one as a member of the RIAA who have issues a DMCA take-down request, and on as the owner of GitHub which has acted upon that order.
1) Downloading videos to use later in editing for analysis pieces / editorial videos.
2) Download videos that may disappear in a "No longer available" blackhole.
The 2nd is incredibly useful and telling when you go back, look up videos by their ID, and see what has been deleted. I'd also recommend changing the OUTPUT format to include the channel name or Twitter account, so you can easily see if an account or channel has been censored/banned.
Watching a youtube video on a slow (1k) DSL has become entirely impossible because a few years back youtube (seemingly) stopped buffering the entire video, so you can't start the video, pause, go do something else and watch it once it's loaded. Not only that, but it really does not handle either temporary disconnections or high latency spikes well.
AFAIK, (good) browsers still give you the direct links if you know where to look, but the slow boiling of the frog is really evident with things like hiding View Source and such (often under the guise of "usability".) The demise of youtube-dl, along with Google's pushing of their proprietary protocols and other continued user hostilities surely paints a sad picture for the freedom of the Internet...
May have to go back to spotify or offline mp3 files, I don't miss having to manage my own playlists tbh.
Some people would argue that optimising YouTube for the bottom 1% of Internet connections makes no financial sense, but I have gigabit fibre and YouTube stutters. It automatically upgrades to 4K videos (of course), but its buffering algorithm is pared so close to the bone that it can't handle high bandwidths as well.
Engineers at Google get bonuses for shaving 0.1% off of something, because at their scale that could be millions of dollars saved.
Hence protocols like HTTP/3, which exist almost entirely to optimise some Google backend by single-digit percentage points.
YouTube has had every last percent of "inefficiency" squeezed out of it, to the point where lots of users have a degraded experience.
Google famously doesn't care about user experience at all. They care about costs and their own internal KPIs, which are all tied to advertising revenue, not "video playback smoothness".
This is why Firefox was 5x slower on YouTube for years. This is why Google famously has next to zero "customer support", even if you pay them. They don't view you as a customer. You're the product.
Firefox users click on ads less. Firefox users tend to have adblock. They're not good products.
Similarly, Google is fighting a turf war with the likes of NetFlix and Apple for advertising eyeballs, so they do not want to ensure that Apple TV can play back YouTube in the best possible quality. They optimise for Chrome and Chromecast first, everything else a distant second. Got to build that walled garden!
I pay NetFlix the same amount monthly as I pay for YouTube Premium. NetFlix provides support, YouTube doesn't. NetFlix works flawlessly on every device I have, YouTube doesn't.
YouTube doesn't play 4K on my Apple TV 4K! It doesn't play 4K on my flagship Samsung TV! It downgrades my iPhone for 480p even on WiFi!
This kind of anti-consumer (anti-product?) bullshit is why Google needs to be broken up.
The vendor that makes the device, the browser, the search engine, the network protocol, and the advertising platform shouldn't also be television for half the world.
They shouldn't get to degrade the experience to benefit their browser team. They shouldn't get to slow down the experience for a competing browser. They shouldn't get to simply ignore customer complaints. Television broadcasters in most countries have to answer to an ombudsman. YouTube doesn't.
That's too much control that invites anti-competitive, anti-human behaviour. The incentives are all wrong.
> I pay for YouTube Premium
So you're aware that you're the product even if you pay them and you still pay them?
Not that it matters as it's a drop in the ocean.
In a poor town, a shopkeeper may accept barter instead of cash. Their customers might pay them with eggs, and they would be treated like valued customers even if they're too poor to use trade with real money.
It's a matter of corporate culture on how the consumers are viewed. For largely advertising-driven companies their users are products, even if they pay. For largely product and sales-driven companies their users are customers, even if they're in a free tier or a trial account or whatever.
You can make something quite bad on average, and that's okay, you just have to be able to filter out everything you don't want and keep what you want. When CPUs are manufactured, this is what they mean by the "yield". It's the percentage of the product that can be kept, with the rest of the wafer discarded.
Chef's Gallery had a scene that actually shocked me a bit -- this award winning chef was making this deep-fried puff thing that was absolutely perfect. They showed his process, which was to make dozens of them and then plate just the best one for the customer. He never had a knack at all for making them perfect! He was just throwing out 99% of the puffs that he made, using the same technique as anyone else would.
You just have to change your perspective: You're the product. You're the deep fried puff.
If you're an outlier, you will be discarded. You're the bent piece of framing. You're the slice of the silicon wafer that failed the test.
Nobody feels the slightest bit bad about rejecting a faulty product on the production line. No tears are shed. No phone calls are made to the product to see if there's anything the manufacturer can do to fix the situation.
This is Google and by extension YouTube in a nutshell. They're an advertising company manufacturing ad impressions and ad clicks. Viewers are their product.
Viewers on 1 Mbps or 1 Gbps are equally outliers. Both are too weird to cater to, less than optimal, unpredictable, difficult to advertise-to viewers.
Rejected.
Viewers on 1Gbps give off two important signals to advertisers.
1) Probably living near a city to get fiber coverage, which these days is usually a signal of wealth
2) Can afford a fiber internet monthly subscription, also usually not cheap.
This is most likely somebody that advertisers are very interested in getting their ads in front of!
> 2) Can afford a fiber internet monthly subscription, also usually not cheap.
Unless you’re assuming US residents these are not really signals or helpful. In Japan or Korea you can get GBPs fiber for $30. In Eastern Europe you can get fiber for as low as €10 in Romania I think, in western you can get that for 50 in Switzerland.
Meanwhile there are places where it’s not an option at all unless you get into “contact us” price ranges.
This is a peering issue and your ISP's fault, not YouTube's. They can't fix bad peering from their end, your ISP has to do it.
They're too aggressive on their optimizations for mobile networks/devices.
There's even a little popup saying something like "It seems we can connect to the server, do you want to play your downloaded songs instead ?" when there are connectivity issues.
We could have had so many nice things if it wasn't for those things.
It's almost as if there's somewhere a country which acts as a fertile breeding ground for these uncontrollable unregulated monstrosities, to grow really big, and at some point left roaming wild for other countries to deal with "yeah we don't do that here" (by which I mean stuff like false advertising and reasoning like "if it's legal and I make a profit then I get to do it").
I self-host some things because I like the technical challenge, and what I've found is that the effort required to maintain the online presence of the data is quite demanding. Less so if it's outsourced, but without actively 'tending the garden', it will inevitably disappear at least from the public-facing internet.
The ephemerality is also exacerbated by Google / Facebook account terminations.
Personally I've started locally saving anything I think I would want to watch or reference again. When favoriting something or saving it to a playlist, I download it too. I started off with using youtube-dl on the command line and have experimented with the "Import from YouTube" function on Peertube but ended up with a tiny cobbled-together video platform, importer (youtube-dl wrapper), and Firefox extension. I think the next step will be easily saving non-video things - probably recording WARCs?
I did that with only one YouTube playlist, so it wasn't too much work.
I wonder if certain famous YouTubers used youtube-dl for this purpose. How else would you download, edit, and analyze a video's editing or special effects?
edit: I only meant this in so much that it's entirely possible for Google, that controls both Chrome (the biggest browser) and YouTube (biggest online video site) to encable DHCP restrictions on video capture from within the browser for sites (such as YouTube) that might implement it. Given their proclivity towards giving the RIAA/MPAA whatever they ask for, it wouldn't surprise me.
I'm guessing you meant HDCP?
I'm pretty sure that encrypted media sites already use HDCP because my projector glitches for a few seconds sometimes when I first navigate to Prime Video, and whenever I have a secondary analog VGA monitor connected Prime Video will only play in standard definition.
i wager to say that the vast majority of browser extensions and websites that allow downloading of media content use ytdl in the background
John Deere uses it to go after sellers of unauthorized replacement parts for their tractors.
Non-compliance is a felony carrying a 5 year / $500k punishment. Unlike takedown notices, there's no counter-notice opportunity for the receiving party. So if the receiving party believed the notice isn't valid, they'll need to argue that as a criminal defendant in criminal court.
"You shall not download"
Or what? Can Youtube actually sue me for damages, they haven't suffered any. Some videos are lisenced as Creative Commons. I seriously doubt this has any legal force.
- "unless you see a “download” or similar link displayed by YouTube on the Service for that Content." -- If there is a download link somewhere in the comments you're allowed to download using youtube-dl?
- "provided functionality" -- either functionality is available or it's not. Whether there is a button for it on the UI or not is entirely different. (However, this sort of seems to imply that the HTML specs are legally binding?)
- Your internet connection is only capable of sending and receiving packets, and it isn't wrong at all to describe "sending" as "uploading" and "receiving" as "downloading".
- It says that you aren't allowed to (among other things) "display" any Content for any other purposes -- other purposes than what? "For your information and personal use"?
- It used to be that you could just copy video files from your browser's cache directory, or you could use hard links -- then you wouldn't even have copied them technically. (Not sure if that's still possible on youtube -- but again what if you had a browser that generated PNG files instead or or before "displaying" stuff on the screen?)
In an era of fakes and deepfakes, we need tools like youtube-dl more than ever.
Could you tell us more about this process?
If you use those tools, it's really straight forward process of using the tools. I didn't know about Audacity until I had spent most of my time figuring out ffmpeg/ffplay. It was an amateur effort, but I've done a lot of security analysis.
I've been working on deepfake detection for the past two years. I use pytube3 and youtube-dl to scrape hours and hours of footage, not just youtube, but cspan, news sites, anything I can get my hands on. I have ~100 hours pulled so far, at multiple encoding rates.
Facebook recently changed their API and now the best I can scrape easily is 240p. That's insufficient to detect the artifacts the models need to train on. I can only pull 1/3rd of cspan videos I can watch through the viewer because again, api changes. It's a constant cat and mouse.
I'm not exaggerating when I say that if I don't have those wonderful folks out there keeping these tools and test suites up-to-date, this project is massively disadvantaged.
I don't have any public release assets at the moment, but you can read about Siwei Lyu and Hany Farid, who are leading the charge in deepfake detection.
http://www.cs.albany.edu/~lsw/
https://futurism.com/expert-online-disinfo-covid-black-lives...
I produce/host a few weekly news shows focused on developer-centric news and updates and have a keyboard macro setup to run youtube-dl against a text file with URLs not to download the videos, but the thumbnails for videos, so I can use them in the on-screen graphics when talking about a specific story. Before I scripted that solution, the process of having to manually extract the thumbnail for any video I was highlighting was a major PITA.
Also, being frank, youtube-dl is significantly better than the official YouTube API for downloading past content from channels I own/manage. It’s faster and a lot more scriptable. I have automated scripts set to watch specific playlists or channels and auto-download stuff for archival purposes — and again, this is content that I either own or that exists on a channel where I’m one of the admins.
It’s unfortunate that the test suite had links to commercial music — we’ve seen in past RIAA litigation that that is enough to go against the argument that this isn’t encouraging download of copyrighted content.
But as a tool, for not just YouTube but so many other services, it’s invaluable even in non-data hoarding/grey area or straight up infringement scenarios.
To be clear, I’ve often used youtube-dl to infringe (as have the vast majority of its users), but that’s my choice/fault. The tool itself has plenty of non-infringing uses. I just wish they’d either linked the test file to another area or used other examples.
I've never understood why they don't offer a download option for such videos: the likes of youtube-dl is the only way to get a copy, and it's perfectly legal.
Likewise for public domain videos.
youtube-dl et al are niche enough that it doesn't put a real dent in their ad business, but making it more convenient with a Download button might.
That's certainly something they can do; they just don't do it.
Compare movies on VHS tapes, which all roll ads before actually playing the movie.
- When you fast forward VHS tapes you still see the adverts. When you time seek a digital video file you don't.
- It's harder to do targeted advertising (since you no longer know if the downloader is also the viewer)
- Adverts baked into a video file can't be changed out later for newer adverts after the user has downloaded the file
- There's no way of having users "click" those adverts, let alone report back to Google that they've engaged with it
- Some of those aforementioned points might also have a knock on effect with how advertising is charged, which would mean Google would need another pricing model
...and all of this is before you even address the technical issues of video files having multiple different bitrates and formats. You could probably generate a new file on the fly but that would be extremely computationally expensive. So they'd rack up more costs hosting the service as well as lose money on advertising.
Honestly, I can't blame Google for not supporting a download option.
But I grant it's technically possible to do with a single MPEG file, just not trivial. You would need to prepend the advertisement to the video bitstream (re-encoded with same parameters such as width, height, FPS, intra picture interval) and then do the same for audio, while ensuring the video/audio synchronization is kept intact (MPEG edit lists are not well-supported by players, such as by ffmpeg), while preferably not re-encoding the actual video. A bit more complicated to do the insertion in the middle of the file.
It's possible to do, but I don't think it's something that any off-the-shelf tools actually do.
The adverts are a different file to the video files. So you need to either merge them ahead of time or dynamically splice them in real time.
Ahead of time: Now you need to not only have a video asset for each bit rate and video format but you need to multiply that with the number of adverts on any given day. This would result in thousands of files, maybe even hundreds of thousands once you take targeted adverts into account. And they'd need to rebuild their entire catalogue whenever an ad campaign ends. Clearly that isn't going to work long term.
Real time: Option 2 is to stream the advert then follow immediately with the video content on the same HLS stream. This is much more achievable than option 1 but you are then running the streams like a live TV service where you're dynamically splicing content into existing streams. It isn't difficult to do per stream but it is extra processing compared with the existing set up. The real problem lies with scale. A TV broadcaster might do this with a dozen to a few hundred channels, each with contracts ranging from thousands to hundreds of thousands. Youtube have millions of streams, each which earn pennies from advertising (and even less if they had to download because of the change in advertising model -- as I also described before). So they'd have to pay more and earn less. The financials simply wouldn't stack up.
Before you comment that YouTube and Twitch offer live streaming services based around the same advertising models, yes they do, but they also operate using the same stack as the pre-recorded videos because those video streams don't need to be spliced by YouTube / Twitch (any editing happens by the content creators before it hits YouTube and YouTube can show the adverts before or even between the videos by injecting them in the browser (ie so the splicing is managed at the front end rather than on Googles servers). None of this is doable with a "download video" button.
Otherwise, this is the same problem print newspapers have; how many readers per copy, seeing what adverts, how many times? This can be modelled or assessed (e.g., by reader response codes / instrumented URLs) fairly readily.
Thankfully plenty of people were also using BitTorrent for things like distributing Linux distributions & creative commons material.
That, again, may perhaps save the day.
The same way governments and bureaucrats fuck up things like Covid responses, because they can't even imagine people not having 9-5 office jobs which are totally suited to WFH, and then blithely implement disease protection schemes failing to account for low income and precariously employed people, many of whom are working 3 jobs to pay the bills (including stuff like cleaning jobs where they work at 4 or 5 different old people's homes) - and then wonder why outbreaks spread so fast.
Googlers have probably forgotten the olden times, when they didn't have Gigabit connections to their pockets and lounge rooms, and don't even remember last time they wanted to "save a copy into a hard drive locally, like a hilarious boomer!"...
But I lived in Mountain View. Decent internet is more or less not available there. Your choices are comcast or DSL.
Maybe they just don't remember the last time they had a waking hour out of the office ... :)
It's clear to me that whomever is making these product decisions has a very different relationship with computing and their data than I do-- but whatever the reason is, I don't think it's because they have a much better internet connection.
Sure, that's why projects like Project Loon were launched by Googlers.
I don't think the CC video thing is even ads related. Just that most videos are not CC and no one thought about creating an exception for CC videos.
(Also if you live in Mountain View, you’re stuck with a Comcast cable modem and a 38mbps max upload speed even when paying for their “gigabit” plan. But you'll only get 10-15mbps during the peak hours of the work day.)
youtube-dl has an experimental --include-ads option =)
I use youtube-dl to view videos on other sites (such as vimeo) that don't display in my browser (due to compatibility or privacy settings), usually product support stuff.
I use youtube-dl to collect evidence about cryptocurency scammers, which often gets removed from youtube by the time the scam starts collecting. By collecting it when its online it makes it possible to hand over to lawyers so they know what to go subponea. At least once a video I saved probably personally saved me from a frivolous lawsuit (attorney fired his lying client after sending me a threat letter which I responded to with an archived video).
I sometimes use youtube-dl to buffer longer pieces which I'm watching which might suffer from infuriating connection interruptions.
I don't use youtube-dl to save copyrighted music (except incidentally), among other reasons: the audio on youtube is often pretty low quality.
- for videos
- for website rendering
AFAIK the video acceleration part is the big problem as the issue was ignored for many years by the browser vendors. For rendering websites most browser should do just fine, some hardware-browser combinations might be exceptions though.
I also wonder what you mean by Firefox has no user profiles, because Firefox has a profile concept too. Be default it doesn't ask you, but when you start it with `firefox --ProfileManager` you will see what I mean. I don't like messing with the Profile Manager every time, so I just have different Icons for starting different Profiles (e.g. Music --> Spotify, Movies --> Amazon & Netflix, etc.).
Currently there are just two websites for which I have to use Chromium:
1. Geforce Now, officially Linux is not supported, but with Chromium it does work when you change your user-agent
2. Binance, for whatever reason their login capture doesn't work with Firefox
On the profiles part: I am aware of the --ProfileManager option. But the icon is new to me. Should be worth trying. Thanks. Work uses G Suite so I don't know how Meet + some other tools react with Firefox.
chrome://gpu/ -- here you can see the current status of your GPU Acceleration
chrome://flags/ -- search for 'GPU' or 'accel'. This is the place where you can override some related settings, but be aware, that this might break your browser, so better backup your browser configuration directory before changing anything ;-)
Hardware is a HP T610.
h264 is still the only widely supported in-hardware decodeable codec out there (cheapest for the client) but google serves webm by default (cheapest for them and built in-house)
ytdl-format lets you choose
And I couldn't care less about VEVO and other copyrighted material, which, I wish, could find home in a completely different platform for all I care.
Having said that, can we please stop pretending like any of us weren’t using this to infringe copyright? Most of us weren’t distributing anything, but I think most of us were aware this was at the very least a grey area, if not outright infringement.
I’m a huge fan and user of youtube-dl and think this action from the RIAA is ridiculous (though not surprising — I’m a little surprised it took this long) and that the law over this stuff is absolutely bonkers — but the commentary and false pretenses about not just how we use the tool but the purpose for why the tool was built is incredibly disingenuous.
It was built and designed to download content that the creators or sites that host the content either didn’t want people to download or outright didn’t allow. The program has support for username/passwords for TV Everywhere SSO’s for premium services. There are ways to tunnel in via a proxy to avoid region block downloads. Again, I’m a huge fan, and I’m someone who absolutely used these features, but let’s not pretend the purpose wasn’t exactly what it is.
That doesn’t make the RIAA’s actions any better or anything — but I really dislike pretending like we weren’t all using the program for the exact purposes the complaint laid out, or that they use case wasn’t the primary reason this tool existed.
Just because someone isn't an RIAA member doesn't mean they don't have copyright over the content they upload to YouTube. There are some YouTube creators that would be totally fine with you downloading their content but you can't assume every creator is the same. Just because a user filmed, edited and uploaded some video from their own house doesn't mean they have any less rights than music or video from major labels or Hollywood studios.
Note, I'm not arguing copying small-label stuff is OK, I'm rather more OK with copying RIAA stuff.
AFAICT, if a DRM prohibits fair use, breaking it is legal. In some jurisdictions. Or at least it should be. That should be enough of a reason for youtube-dl to exist.
Again, if anyone is guilty of anything, it's the users (like me) who use the program, not the program for merely existing. And frankly, since I don't distribute the content I download or profit from it in any way, I welcome the RIAA coming after me, just as I did 20 years ago when I was a high school student using Napster on her dial-up connection.
So how was I infringing copyright by downloading something from a public website where copyright holder left the content publicly accessible to anyone?
I'll give you that it may or may not be against ToS of the distribution service, but that's all. But it's not a distribution service causing the ruckus around youtube-dl, but the copyright holder, so that's kinda beside the point.
I'm not pretending if I'm not violating, even if I'm downloading the copyrighted videos.
Most countries in the west allow it in the American sense. Only a very limited few countries allow it in the sense of Spain and Switzerland.
I had a feeling I was using youtube-dl to avoid ads (not blocks), but after some retrospective, I don't think I would have seen ads on most videos I watched from youtube and I think most were freely available videos, often embedded on author's pages - except people have an understandable habit of hosting them on youtube.
To conclude, I would disagree that we are all using youtube-dl as complained.
https://www.playon.tv/blog/playon-legal
As long as you don't rebroadcast copyrighted material, you should be fine.
For example, the first thing he did after I helped him set up OBS was to fire up a game + Spotify to make a live stream. I had to explain to him why he's not allowed to stream music like that. He doesn't do it now, but there's still a decent chance his channel that gets 5 views per video will get struck or banned for a similar misunderstanding in the future.
He doesn't know much (under 10) beyond using the Windows Game Bar or OBS to upload something and doesn't have a bunch of spare storage, so he doesn't keep copies of videos anyway. I download them so if he gets banned I have an archive and can help him set up a new channel on another platform.
I know it'll be easy for someone to say "do X, Y, and Z" then, but the reality is that it doesn't take much for it to become "not fun" for younger kids and the opportunity to encourage them to learn valuable skills like video editing, etc. are lost when they lose interest.
Speak for yourself. I used it to pull down youtube videos to watch on a 14 hour plane ride over the pacific.
(I’m not really a stakeholder tho so who cares what I think)
Intellectual property law is about controlling what you can see, and do, with your own eyes, ears, and hardware.
If you stream a video on YouTube, YouTube can arbitrarily take it away from you whenever they want. If you download a video from YouTube, it's yours to keep. Tools like youtube-dl take away the power YouTube has over you, and that's an unacceptable loss to the ownership class even if they suffer no financial losses.
... funds OTA television. Also funds YT.
With OTA, it is okay to record OTA broadcasts using a VCR, so I'm not sure why youtube-dl would be different.
But really, if the VCR were invented today, it would be made illegal.
“I say to you that the VCR is to the American film producer and the American public as the Boston strangler is to the woman home alone.“ - Jack Valenti, former MPAA president, in 1982, explaining to Congress why the VCR should be illegal
It worked brilliantly, back in the days of analog cable TV.
ReplayTV was sued and went out of business before it could be fully adjudicated. Dish was sued over its Hopper DVR that auto-skipped ads and ultimately had to settle. TiVo won some of their lawsuits but because it didn’t do automatic commercial skipping of the recordings, was spared her wrath of the networks like Replay was. (TiVo was ultimately ruined by the cable companies who sought to introduce their own inferior DVRs that they could charge a monthly fee over.)
[1]: https://en.m.wikipedia.org/wiki/Sony_Corp._of_America_v._Uni....
I remember that dissent (authored by the late Justice Scalia—RIP). Here’s the citation in question:
> We came within one vote of declaring the VCR contraband 30 years ago in Sony [v. Universal]. The dissent in that case was driven in part by the plaintiffs’ prediction that VCR technology would wreak all manner of havoc in the television and movie industries.
> The Networks make similarly dire predictions about Aereo. We are told that nothing less than “the very existence of broadcast television as we know it” is at stake. Aereo and its amici dispute those forecasts and make a few of their own, suggesting that a decision in the Networks’ favor will stifle technological innovation and imperil billions of dollars of investments in cloud-storage services. We are in no position to judge the validity of those self-interested claims or to foresee the path of future technological development.
Recorded TV and cut out the ads.
Dish’s Hopper actually DID do ad-free recordings and it was sued too (but there was eventually a settlement), not just for skipping the ads but for distributing the same copy of a recording to multiple users over a server. Cablevision, likewise, was sued (but settled) over the fact that it had a cloud DVR network that allowed the same copy of a recording to be available to multiple people, rather than storing X-Copies of the recording on their servers.
This is how John Deere got away with prohibiting farmers from repairing their tractors, etc. This is why time-shifting and format-shifting were just fine for OTA/cable television/CDs/etc but are practically nonexistent today.
> In 1984, the Supreme Court ruled that Sony could continue to sell its Betamax videocassette recorder, overruling the U.S. 9th Circuit Court of Appeals judgement that held Sony liable for consumers’ copyright infringement.
https://consumerist.com/2014/01/17/on-this-day-in-1984-the-s...
Same s*, different decade.
On the other hand, I'm no lawyer, but I think there's a (possibly bad) argument that youtube-dl does not violate 17 USC §§1201(a)(2)(a) because it does not circumvent any technological measures to control access to any content. The video is transferred entirely in the clear to any anonymous users that request it. Youtube-dl makes no false claims or breaks no cryptography or encryption. What measure is in place that is being circumvented?
https://en.wikipedia.org/wiki/Sony_Corp._of_America_v._Unive....
https://github.com/github/dmca/blob/master/2020/10/2020-10-2...
There is legitimately some obfuscation happening:
https://www.jwz.org/blog/2013/06/youtube-download-counter-co...
...yes, it is? What difference are you imagining?
If the code snippet that downloaded infringing materials didn't exist and the user had to find the parameters themselves, that's fine. But it does exist and was included alongside a tool that makes that recording possible.
A better analogy would be like, if you had a video store that had a machine in the store that allowed people to make copies of a video tape using a duplicating machine. If someone wants to buy a duplicator for home use and use it in their home, that's one thing (though there was litigation around that too), but if you go to a store and can plug in a video tape -- and the example in the store is a copy of a Disney movie -- that's sort of what this is.
(Photocopy shops and libraries often have fairly strict rules about what kind of content can be photocopied, for what it's worth. The guy at Kinkos at 3am might not care that you're copying an entire textbook on the machine, but if you go to Office Depot in the daytime, they definitely will. Kinkos was sued and lost over a photocopy case in regards to excerpts/packets/anthologies for textbooks.)
That's what's happening here. Github is selling (for free) recording machines.
If you have the recording device, then you can use it to record video that gets transmitted to you. But there isn't a place you can go to have the video duplicated for you. You do it yourself.
Downloading a video is same as recording it to a VHS and serves the same purpose.
They had no alternative other than not having tests for that particular functionality. The tests were testing youtube-partner specific functionality: the procedure needed to download content from some youtube partners is different.
Moreover, the tests just throw away the downloaded material... it would be entirely reasonable for a court to conclude that there is no copyright interest involved in that activity at all and would be extraordinarily unlikely that any court would find it to be anything but fair use if they did conclude a copyright was involved at all.
If it hadn't been the test cases the RIAA probably would have dug up some old forum post advocating infringement by some occasional contributor. They were always going to argue something.
In retrospect, it would have been much better to link to an external repo or site or source for the test file, rather than to have that as part of the GitHub repo.
ETA: you’re right that they were always going to go after something, but the action for GitHub to take down the repo wouldn’t be arguable if the test file wasn’t in the repo. It’s possible they would have gone to issues/comments within the project (another reason, in retrospect, to host that stuff separately), but in this case there was stuff in the source code that the RIAA can reasonably argue would lead to infringement.
Plus, the unit test IIRC only downloaded a 100KB (thus less than one second) chunk.
The first two elements have a mens rea implication behind them: you have to demonstrate the intent of the developers. That RIAA's strongest evidence of intent is buried in a unit test, and this seeming intent flies against the evidence in the far more prominent README that is discussing its unsuitability for infringing content. Furthermore, they can also demonstrate from their own practice in history that they refuse pull requests and close issues were people are clearly trying to use it only for infringing use cases.
In a normal, procedurally valid, DMCA the only harm to the host for not following the procedure is that they lose the safe harbor.
In this case because it isn't a question of infringement there isn't a safe harbor at all.
In either case a company can choose to take something down because it wants to reduce its exposure to litigation.
I'm disappointed that even after acquisition github has continued with the beyond-industry-standard level of aggressiveness with takedowns. But I don't see how you conclude that the anti-circumvention complaint makes github more in the wrong for complying.
For that particular instance or permanently?
The DMCA gives hosts an option to have legal immunity for each specific case of infringement, to gain the immunity they need only take the content down at the cost of outraging a potential customer. They don't have to take the option, however.
It is common in industry for hosts to simply discard obviously invalid complaints particularly if the target is high profile. The legal immunity isn't very valuable if the complaint is baseless-- (sure, there might be frivolous litigation, but that is always possible).
It also seems pretty common for hosts to allow the target of the complaint to counter-notice in advance of taking the material down and then just skip the takedown, or they take it down but restore it immediately on counter-notice (youtube itself does this, or at least did when I was hit with a spurious dmca complaint there years ago). Both of these procedures don't follow the letter of the law and arguably cost the provider their safe-harbour. OTOH, almost no DMCA complaints are actually valid per the specific requirements of the statute, so maybe they don't actually lose their safe-harbour.
I can't imagine it was the only time such a thing happened, and that alone is an argument for simply ignoring those take-down requests if possible.
What you said is clearly correct, but I don't think it's reasonable to expect any company to insert themselves in the middle of these disputes. At scale it's impossible.
Imagine you run a forum with millions of members and tens of thousands of posts per day. You get a stack of DMCA takedown requests every day. Do you think it's feasible for you to scrutinize each and every one, and take a moral stand on those you deem 'not procedurally valid' ? How much do you think that might cost? What happens when you call one wrong and the law comes to YOU?
For anything to really change, it's clear someone will have to go after the RIAA.. (EFF maybe?)
At one point Microsoft was issuing DMCA takedowns against (then) openOffice.org mirrors, allegedly on the grounds that any large file with a name that contained the word 'office' must be a 'pirated' version of MS Office. Taking these takedowns at face value would have ended the openOffice.org project and absolutely could not have been respected in good faith.
These kind of abuses are ongoing from multiple parties, including bots, copyright trolls, and crazed ideologues who file completely fraudulent DMCA takedowns to force anonymous personalities to disclose their identities. There is no alternative but to consider DMCA takedowns on legal merits because so many of them have no merits whatsoever.
Having the funding to do this 'at scale' is just as much a cost of doing business as paying for power and connectivity. If hosting platforms don't want to do this, then they should purchase an amendment to the DMCA to impose criminal sanctions on entities that serve fraudulent DMCA takedowns.
I want to chew Microsoft out for this one, but the US-based copyright mafia have built themselves an unbelievably vicious draconian set of laws that prevent any sort of challenge, including those coming from scientific inquiry or examination. Anti-circumvention is the most world-hating corporate-owned-world flaming garbage that could be devised, and violating it comes with unbelievably mercilessly cruel criminal penalties.
The menace of this threat has silenced science & speech, has prevented mankind from examining the world about them, learning of it, & discussing it. We have outlawed knowledge, outlawed idea, literally criminalized knowing something about the world with steep jail time. This is a farce, of the highest order, one of the greatest shames the law has done unto itself.
Great recent thread from Doctorow on this: https://mamot.fr/@pluralistic/105090187888073250
I also recommend Bunnie Huang's "Why I'm Suing the US Government", https://www.bunniestudios.com/blog/?p=4782
Anti-circumvention laws must be stopped.
> Like the parable of the frog in the well, their creativity has been confined to a small patch, not realizing how big and blue the sky could be if they could step outside that well.
That was posted in 2016; does anyone know what happened with that lawsuit?
I completely agree, on the other hand, that a takedown notice issued knowingly without proper grounds should be penalised severely.
Except, by choosing to honor it, they decided that it is legally valid.
The appropriate response to a copyright claim against material that the claimant clearly does not own is 'unless you can prove that you own this material, come back with a court order.'
Regardless, I think they takedown without giving the "DMCA" complaint much scrutiny because the risk of penalty for NOT taking action is greater than just taking down the repo(s) risk-free.
Github's ToS: which says "we have the right (though not the obligation) to refuse or remove any User-Generated Content that, in our sole discretion, violates any GitHub terms or policies." ( https://docs.github.com/en/free-pro-team@latest/github/site-... )
I have to say it IS strange they took down content based on a letter that doesn't prove or even really suggest copyright infringement has taken place (only that it COULD, as a result of using the software), and their own ToS says "There may be legal consequences for sending a false or frivolous takedown notice." ( https://docs.github.com/en/free-pro-team@latest/github/site-... )
The headline on the article should reflect the fact that Microsoft/Github pulled youtube-dl offline by choosing to respond to a DMCA takedown order that is not procedurally valid. The privatization of law enforcement through DMCA takedown orders does not extend to alleged circumvention tools; it includes only copyrighted material that is allegedly 'pirated.'
The RIAA is engaging in an abuse of process that Microsoft has no obligation to cooperate with. The fact that Microsoft didn't tell the RIAA to go Disney themselves means that 100% of the blame lies at Microsoft's feet. No exceptions.
. .
That said, the youtube-dl maintainers should file a bar association complaint against the RIAA lawyer(s) who wrote the DMCA takedown. Filing invalid motions is an ethical violation that should result in sanctions.I for example, use it to download large amounts of material for when I am offline while sailing. Programming tutorials to learn some useful stuff while offline, entire university courses on interesting topics, etc. There aren't many practical ways for me to do it.
Have you ACTUALLY tried to download? I am using latest Chrome on Linux and I don't even have the option. Downloading to the phone is not a workable option for me, I need to be able to see it on laptop screen and I need a lot of it for weeks of being offline. And couple other problems like videos ceasing to be available for whatever unknown reason, suddenly.
But all that doesn't go to my argument which is that YouTube, not myself, may not see your use case as legitimate since they provide a subscription service which enables offline viewing. I doubt that because the way they have implemented it, for whatever technical, contractual or just on whim reasons, may not meet all your specific needs they see you bypassing controls to implement in your own manner as legitimate.
I know it's a bit of a childish argument for me to make, but I think ultimately it comes down to money in this case.
Can they takedown my web browser and operating system too? These are general purpose tools.
It's sad that the conversation is all about justifying its use - I think the takedown is absolutely unjustified.
GitHub should have let it go to court.
I hope they put it up onn a Github-like site outside the US.
https://www.instagram.com/p/CFyy7Csjpl8/?igshid=s29gkfosrt2q
https://www.youtube.com/playlist?list=PLBV9SKQZzAWuIXspogOq3...
Sometimes I think the scope of what a court is allowed to decide is crazy. It allows new concepts to be manufactured for the convenience of muddying the legal waters. Why the hell legislate around "DRM" and then decide later what that means.
I can simply record my screen with audio (for example using QuickTime Player) and save that as a video. I do that as a last resort if video download software doesn’t work. This method can never be stopped. If I can see it and hear it, then I can record it, with close to perfect quality. And it will get even easier with more user friendly tools.
Sorry, technology wins.
I'm thinking of Widevine level 1 DRM, for example. If I remember correctly, it mandates that all of the decryption has to happen in a "Trusted Execution Environment", which seems to mean a hardware enclave. I think this effectively locks the user out of the whole decryption phase, which means that you don't get to record your screen.
I might be wrong about this, though. Might have to do a little research.
(Funnily enough, just looking for information right now turns up this https://news.ycombinator.com/item?id=21307308 HN post, which does contain some interesting information)
By making it difficult to archive videos I am starting to see more and more low quality screen captures being shared (e.g. on social platforms). A screen capture of a phone camera-recorded video of a phone-recorded TV screen where it is barely possible to understand the spoken words of the actors. That is pretty sad to see in this technologically-advanced world.
If I stream a file for an hour, I tie up a bunch of network resources for an hour. If I can legitimately DL that in 2 minutes, I'm using less net resources for a shorter time. Why in the hell should that be denied because some material is not in the public domain? Completely illogical result.
I watch a lot of long-playing material (I think is non-copyright but WTF do I know?) that I DL to timeshift and watch offline in chunks as time allows. Is that music concert from Poland that's up on Youtube copyrighted? That amateur history/science/education channel that asks for Patreon help? Screw RIAA (again)
> The clear purpose of this source code is to (i) circumvent the technological protection measures used by authorized streaming services such as YouTube, and (ii) reproduce and distribute music videos and sound recordings owned by our member companies without authorization for such use.
(i) Does YouTube have "technological protection measures" that prevent downloading? In the case of the AACS controversy, this was a bit more clear-cut: there was a secret key that was cracked, but what exactly is being "circumvented" in this case?
(ii) Nowhere does youtube-dl advertise that it's to be used to "reproduce and distribute" without authorization.
I feel like this is a YouTube ToS violation, not a DMCA-strike-worthy copyright violation.
The README file doesn't encourage any downloads of any copyrighted works. All examples included in the README point to their own example video. The only thing that comes close to this is that they say that they will not support any services that are violating copyrights. The copyrighted content is used in test cases, that are not meant for users to see.
> (i) Does YouTube have "technological protection measures" that prevent downloading? In the case of the AACS controversy, this was a bit more clear-cut: there was a secret key that was cracked, but what exactly is being "circumvented" in this case?
What's being "circumvented" is the encrypted URL to the content. But it's not really circumvented, YouTube decrypts it with javascript. All youtube-dl does is that it interprets the javascript that's on their website, just like a web browser does.
> (ii) Nowhere does youtube-dl advertise that it's to be used to "reproduce and distribute" without authorization.
Correct. Not sure what "reproduce" means in legal speak, but youtube-dl doesn't even have any functionality to distribute anything.
And I use youtube-dl to download content from other sites, like nrk.no, svt.se and bbc.co.uk.
Why do companies like MS, Google, Amazon etc acquire smaller companies if they cant protect the values these small companies stand for and made them attractive in first place.
It's the RIAA: Racketeer Industry Association of America.
Either way you can host git yourself (or just use git format-patch) there's not a single thing github really does for you other than integrate a bunch of self-hostable tools behind a flashy (and increasingly less usable) web GUI.
I guess it's time to do sudo apt install youtube-dl
I'm so sick of hearing this "investigations want to be free" nonsense from law enforcement. Just because they want to pirate their investigations without paying they think it's ok for content creators to go hungry?
Get a job hippies.
I write my own downloaders. The one I use for YouTube is currently less than half a page of shell script. I never download VEVO or other videos that have enciphered signatures (most videos I encounter on web pages don't), so there is no need for Python or any other slow scripting language and fiddling around with Google's Javascript player. Funnily enough, my scripts hardly ever break. When they do break because of some YouTube change (only once so far), I can fix them in a fraction of the time it takes for the yt-dl project to respond to user complaints and make necessary fixes.
I doubt the press nor anyone else really needs yt-dl if all they are doing is downloading unprotected videos.
Even if someone wants to download VEVO or other protected ones, they don't need youtube-dl to do it. It is trivial to get the download URL for these videos with any "modern" browser. For example, in Chrome, open Developer Tools Network tab, type "videoplayback" in the search box, copy one of the URLs (alt-click) and remove the &range= parameter from the URL.
I wonder if the RIAA will ever go after persons trying to make money by offering anti-circumvention services, e.g.,
I find all those websites trying to sell so-called "video downloaders" very annoying.