Hacker News Reader

Comment by toomuchtodo | Hacker News Reader

All of it has already been archived (EDIT: thanks to the hard work and quick response of ArchiveTeam and /r/DataHoarder).

[edit: link to Archive Team project-specific page removed to reduce excessive load; replaced with Archive.is link below]

https://archive.is/D1Ail

View on HN

Out of curiosity, if Berkeley requests that the Archive Team remove their videos, I assume they would comply? There must be a reason they want their videos taken down, and could presumably use the threat of legal action to have them removed from any archives? Why choose to take down videos being hosted at no cost on YouTube if you're willing to allow them to be "pirated" (aka preserved) elsewhere?

I don't speak for Archive Team or Archive.org. Archive.org has DMCA exemptions but will comply with robots.txt access restrictions post-archival.

I operate independently as a digital archivist, providing support when possible. I will keep my own copy in cold storage as long as I'm alive, outside of the US jurisdiction but accessible globally.

The reason they're being removed is ADA subtitle compliance and the cost/effort associated with that (cheaper to purge/remove access rather than comply). Is UC Berkeley going to pursue people attempting to preserve knowledge without their consent? That's up to them. They would, of course, be unsuccessful.

For crying out loud. I understand the importance of accessibility, but when complying with mandatory accessibility regulations results in everyone losing access to educational information... well, everyone loses.

Since you can't beat regulation, the real course of action here should not be to archive the videos where nearly nobody will find them, but rather to elicit donations to produce the subtitles. Also, if the content is so extensive, it should probably find a home on a dedicated site other than YouTube which may one day also disappear. It is such a shame to have educational resources essentially become part of the deep web - your average user can find this content on YouTube, but has very little chance of ever discovering it in an archive.

If Berkeley has not already postponed to the maximum extent possible, I would happily donate to the cause. I fear it's likely too late due to a deadline to comply.

> but rather to elicit donations to produce the subtitles

I intend to engage someone at Youtube/Google to use their internal automated subtitling system and open source/release the captions along with the existing content [1]. (Why UC Berkley didn't use this, I'm unaware)

In the event the above is not possible, I'll roll my own when time permits (voice recognition with distributed human verification).

> I fear it's likely too late due to a deadline to comply.

Agreed. Hence, archival.

[1] https://support.google.com/youtube/answer/6373554?hl=en

Automated subtitles without human review of every spoken phrase likely doesn't comply with regulation. It would be very poor quality to have anything transcribed incorrectly.

They are being taken down because the Justice Department is threatening Berkeley using the ADA because the videos don't have subtitles. Once they are out of Berkeley's control, hopefully all parties will just move on, point being made.

If you looked at the "Downloaded" column on that link most of the CS courses haven't been downloaded. Also when I click on a link like I am redirected to youtube where the content is as of now still available. Am I missing something?

[edited out my comment to reduce the noise -- see comments below]

No, the latest statement is that they're still missing a lot of the youtube videos.

"""Most of the Youtube videos on the channel right now are NOT on archive.org. I have asked multiple times in the comments and even inboxed the guy who made the original thread about whether archive.org actually had everything and just hadn't uploaded it yet, and got no responses. If anyone knows that these have in fact been downloaded already, please let us know. But just look at archive.org, there's just a PORTION of what's available on Youtube right now. """

https://www.reddit.com/r/DataHoarder/comments/5z2499/many_of...

Check the top edit on that post; All is accounted for.

Could you link to it please? I'm not able to find it :(

From the top of the Reddit post you linked to in your comment:

"Edit: Everything has been accounted for and is in the process of downloading, thank you to everyone who helped, especially lawpetex who backed up the full amount of all the Youtube and iTunes content!"

Weird. That isn't showing for me :/

Thanks anyway

94 videos at this point - only a small fraction.

The current status seems to be:

>"Edit: Everything has been accounted for and is in the process of downloading, thank you to everyone who helped,"

I wish I had known about this issue earlier, I would have liked to have helped.

Join #archiveteam on EFnet IRC.

Ah neat, thanks.

Thank you, toomuchtodo. This did seem like the sort of thing that just had to have already been done.

Archive Team seems to be down (maybe from all the requests). If you really value these videos, I'd still download them onto your own computer instead of relying on a third-party hosting. As others have mentioned, youtube-dl is the way to do it and super easy, e.g.:

  youtube-dl https://www.youtube.com/playlist?list=PL-XXv-cvA_iBDyz-ba4yDskqMDY6A1w_c

Archive Team and Archive.org are not the same entity. Archive.org contains the entire UC Berkeley course corpus, provided by Archive Team and /r/DataHoarder/

You are free to help seed the torrents of course containing all of the content.

Thanks for clarifying. Yes, seeding lectures would be the best way to contribute while still maintaining a personal copy.

https://archive.org/details/@schule04?and%5b%5d=subject%3A%2...

(If you go onto one of the course pages above, there's a download option to grab a torrent.)

Can you point to the exact page on Archive.org that has all of the courses?

This appears to be the Berkley page that links to all the available courses: http://webcast.berkeley.edu/series#c,s

I've checked the various Archive.org links posted so far, and only see a small portion of the content. For example only the 2008 and 2010 versions of CS61A & B appear to be archived.

Has anyone been able to use youtube-dl to archive the CS61B-2016-Spring? 'Computer Science 61B, 002 - Spring 2015'

https://www.youtube.com/playlist?list=PL-XXv-cvA_iDD4nnsfVIq...

Unfortunately this content appears to be throwing HTTP 500 server errors while downloading according to the verbose output.

If anyone has that content I'd be interested to know what worked for you.

I have no clue why that error is happening but I was getting the same result. Skipping the first video got it to run for me:

    youtube-dl --playlist-start 2 "https://www.youtube.com/playlist?list=PL-XXv-cvA_iDD4nnsfVIqPFORTgZi9xRp"

For whatever reason the first video is downloading fine for me [edit: no it's not, see below]. Try again maybe?

If it still refuses to work for anyone else, I have like 80KB/s upload (ADSL2+ FTL) but I'm fine with uploading it.

EDIT - HERE YOU GO: The highest quality video (format code 136) is WORKING, but the highest quality (189k) audio (format code 251) is what's failing. So fall back to the next highest quality (132k; format code 140). So '-f 136+140'. I've __successfully__ downloaded with these settings.

Even I don't know what was causing it to be failed. What worked for me was downloading starting from the 3rd video and then downloading the 2nd with the help of a firefox plugin.

3.1 TB!

> Resource Limit Is Reached

Unfortunately the site seems to be overloaded.

Reply on news.ycombinator.com