Show HN: Khan-dl – Khan Academy Course Downloader
github.com
github.com
I may never use it again, so simple garbage collection script does it for me, but it's much better than going to youtube/another source and find something "author made video private", "channel was removed".
Pornhub and its sisters recently dumped pretty much everything by unverified users thanks to an NYT hit-piece. Depending on your interests a lot of content went goodbye.
Youtube frequently delists music videos.
Creators sometimes remove old videos for various reasons.
Shit like this is why the cloud is not to be trusted.
Exactly.
No such thing as "the cloud" it's just other people's computers.
I think you mean companies/YouTubers themselves - YT doesn't remove regular videos on their own very often. The content being deleted is still a problem and reason to save it though - many externalities might be reason for the uploader to remove videos (legal, purging a Google account, etc).
https://www.nytimes.com/2020/12/04/opinion/sunday/pornhub-ra...
Indeed, but they through the baby out with the bathwater on that one. A tiny percentage of the content they deleted met that criteria.
It had over a thousand videos from what I remember in 2015. Now there's barely ~600 left. Lots of them got deleted due to copyright, the Vox Adpocallispe (thx Carlos Maza) and cancel culture.
I just downloaded all the remaining video using youtube-dl but I should have done that before, now it feels I lost something important in those memories that I will never get back.
Here's the command I used, took me a while and might save other people a few hours:
pip install --upgrade pip
pip install youtube-dl
youtube-dl.exe -i -f bestvideo+bestaudio --merge-output-format mp4 "https://www.youtube.com/playlist?list=XXXXXXXXXXXXXXXXXXXXXX..."
Obviously if you're not on Windows remove the .exe
This will download an entire playlist at the highest video and audio quality and will not crash if a video has an error.
EDIT: YT does not send a 0byte file for deleteds, only a "video unavailable" error.
Not the same as bestvideo+bestaudio which will merge separate sources.
Another example was Walter Lewin's physics videos all got pulled by MIT from Open Courseware (but are available on YT at least).
Not a comment about the parties involved, just saying there's external factors sometimes that will vanish a video.
https://www.insidehighered.com/news/2017/03/06/u-california-...
https://www.insidehighered.com/news/2015/03/10/despite-mits-...
It's a shame, but any other solution feels like it's a broken stair just waiting for someone to trip over it.
- The organization signaling to stakeholders that they condemn and take appropriate action against behaviour.
- The organzation engaging in damage control, minimizing future exposure of events linking back to the organization.
- Penalizing transgressive behaviour in a professional setting, through sanctions and revocations of merit, reputation, licensing or employment etc.
Actually, breach of contract could lead to lecture material not being legally available, if a lecturer has sole ownership of such. It could also be that future use of say, recorded lectures, involves compensation to the author. If an organization want to cut ties to the author, or just withhold compensation for a time, they might refrain from using the authors material.
https://www.insidehighered.com/news/2015/01/23/complainant-u...
This sort of thing really bothers me. Accessibility should absolutely be supported where feasible, but never to the point where it prevents releasing content in the first place.
Regardless, no one said Berkeley had to take the videos down. They chose to do that on their own because it was too “expensive” to make them accessible.
However, I did not see Berkeley turning down federal funding, which is in part why they are legally required to make their courses accessible (Section 504, for those wondering). Funny how that stuff works.
Regardless, a lack of more effective means for enforcement does not absolve Berkeley of failing to follow the law, especially when they had internal resources to make their material accessible and chose not to do so.
Edit: apologies, misread your post and that is an interesting approach. Another way is to make the university pay back a portion of the federal funding they received, up to whatever amount they estimate it would take to make the materials accessible.
1. Release content not due to requirement but for general good. 2. Get sued because it's not captioned. 3. Have insufficient funds to caption so you pull material. 4. Get fined for pulling material. 5. Watch as no one does step 1 anymore.
The question is, if a college professor would like to throw up a recording of their course online for the public benefit, and the school would otherwise be on board, will they be able to do so? Or will the school be concerned about getting sued because the video doesn't have subtitles?
Adding subtitles to many hours of lectures which will be posted for free is not a reasonable request. So of course what will happen instead is the videos will never be posted.
If this scenario isn't actually plausible, feel free to correct me.
It really feels like the Overton window on disability rights has shifted in the wrong direction.
Just because it’s free, the university has certain obligations from taking federal funding. These laws have been put into place in order to ensure disabled people have equal access to these educational systems.
One way to solve this would have been for Berkeley to charge a nominal fee (perhaps for a limited time as “early access” or similar) that could then be used to pay people that can transcribe the audio.
Instead, a lack of imagination and a lack of empathy leads to these resources being removed, a net loss for everyone.
California did not back then, but they now have their own legislation known as the Unruh Civil Rights Act and of course there’s the ADA/Rehabilitation Act.
Here’s the kicker - they have to do this whether it is posted publicly or not. The legal requirement does not only apply to public material, but to ALL material and the material still exists/can be accessed with Berkeley credentials. It also applies to future material produced that is used with students/in classes.
My guess is, they had to make it accessible regardless and did this out of spite. They had internal resources to help with this but chose not to do so/made it really expensive because of the volume due to not doing it in the first place.
It is kind of like Cybersecurity - people don’t like to spend the money until they have a breach, where it’s much more expensive to remediate and their costs would be much more manageable/risk reduced has they done it “right” the entire time.
My mother is a professor in the CSU system and I will say California public universities are beginning to take this much more seriously now, largely because of lawsuits.
The courses are not free but the videos are! Releasing publicly available videos is not a requirement in order to receive public funding, it's something extra they did because they could.
> My guess is, they had to make it accessible regardless and did this out of spite. They had internal resources to help with this but chose not to do so/made it really expensive because of the volume due to not doing it in the first place.
If the subtitles were already available for students internally, that does change things. But the articles posted by the OP made it sound like it was something they'd need to add.
And adding subtitles to hundreds of hours of lectures really does strike me as unreasonably costly, given that this content was not intended for students paying tuition and did not garner any additional funding.
They had an office dedicated to helping professors make course materials, to include resources for subtitling course material and QA processes for ensuring accessibility. For whatever reason, the university did not require use of the office and the professors did not use it even though it was there. The DoJ report I linked above has additional information on that.
I may not be remembering accurately, but I am fairly sure the material Berkeley posted on edX at the time was just recordings of their actual lectures given to on-campus students and/or recordings they were making/were made for online classes (that were paid for). In other words, they were not going out of their way to record these lectures specifically for MOOCs.
Considering that, we should remove overzealous captioning laws and simply require auto-captions and focus our accessibility efforts elsewhere in society (like making more new build houses step free - it's ridiculous that 90% of my friends houses I can't visit).
Even if it happens only once, it may be a really sad event. I had a "go-to" youtube video (nothing relevant in the grand scheme of things, just a beautiful rendition of a famous neapolitan song in the original language, not in italian). I listened very often because it cheered me up. Now it's gone for good and I'm sure I won't ever be able to listen to it. It was not a record, just a lady singing in a public place, obviously recorded with a phone.
Around 10-20% of the older videos are gone... from copyright infringements (music in video), to deleted channels, to "not available in your country). What I hate the most is, that even the video titles are removed, so i have no idea what's gone, just the "[Private video]" or "[Delted video]" in the list.
Right down the memory hole. Your "favorites" aren't even yours.
- 6 out of 71 videos in favourites,
- 20 out of 209 in playlist of videos I consider interesting;
- 7 out of 56 in playlist I keep to give links to other people;
- 15 out of 112 in playlist of funny videos;
- 27 out of 218 in playlist of WTF videos;
- zero out of 89 in music video playlist;
- 19 out of 121 in political video playlist.
Especially when it comes to educational or conference content, those accounts seem to close with higher frequency.
I know you can't answer this question, I just find it frustrating that stuff gets removed for reasons unrelated to the content.
But at least YouTube tells you something was removed, although I do wish they would tell you what it was which was removed. I was an active google music user for the life of the service (RIP) and sometimes I would have a vague thought that it felt like my playlist was missing something but they never actually told you. It was infuriating.
The cloud is ephemeral, do not rely on it.
In my list I have at least 30 deleted videos.
I set up youtube-dl to fetch all videos in my playlists daily at 4am. Digital hoarding is very cheap.
Just because it doesn't happen to you doesn't mean it's not a problem for many of us.
It also depends on the type of content, InfoSec stuff disappears all the time while popular music videos probably never vanish. "Despacitio" is not going anywhere.
I now have a machine specifically set up to download videos from my favorites playlist on a daily basis
We maintain a list of content that various members have archived, such that when content is removed from youtube, people can direct inquiries to contributors who have archived that content.
It's a small way to keep track of what things have been successfully archived, and direct efforts toward at-risk channels.
(You can see the current list of user-archived content here: https://tinyurl.com/v4rpe9w)
Kids can learn from online sources. So why should someone's education be tied to their school district or teacher quality?
I would love to have the local school system provide lab space, athletics, support services and one-on-one tutoring as necessary.
But the bulk should be delivered online. Sal Khan is comprehensive and brilliant, but perhaps someone out there is better at chemistry or art history. Regardless, every kid should be exposed to the "best" teacher of the subject. Why should a child's Algebra education be school or teacher dependent, when the best Algebra teacher on the planet is just 256kb/s away.
Online resources are incredibly valuable as supplementary material but they can't replace the culture and bond of small group learning
In poor performing schools, there are multiple reasons for poor observed outcomes.
None of these need to happen.
Keep small group learning and have adults support learning. But there is no reason for a teacher to explain something when Khan academy has already done it and often done it far better.
>But there is no reason for a teacher to explain something when Khan academy has already done it and often done it far better.
You're skipping the often difficult task of getting the students to properly engage with khan academy.
As someone who is very much in favour of online education, this is still a pretty big unanswered question.
My response to that would be to ask to what extent this is being done in the classroom. Much like in the workplace prior to covid, people mistake physical presence for actually doing work. And if the classroom is then replaced with online learning with the ability to play back lessons and study at your own pace, how do you define "engagement"?
I think my interest in programming as a kid pushed me to online learning and taught me to be independent. Most of my friends barely went beyond what the teacher covered.
I still think that school as an environment where you see and interact with other students and tutors is useful - but merely for a social purpose. Front lessons can't disappear early enough.
How do you force them to engage with the content?
Schools have many impact vectors, forcing content learning is just one of them, and not necessarily a good one, I'm biased into thinking it's actually bad but it's overcompensated by its side-effects: place of gathering, exposure to higher-than-average smart people (teachers), introductions to some branch of knowledge, etc, and of course, forced-content consuming from interested students that probably would get better results from a self-paced learning system like Khan-Academy
Interestingly, you can find bad teachers and bad schools at all points of per pupil spending. Money is an important variable, but not the only significant variable effecting student outcomes.
I think we should radically rethink teaching. We can support much better outcomes for far more kids by reducing the dependance on the quality of your teacher, school and school district.
I think it would be easier to get kids to engage with Khan and online instruction in general than to fix failing school districts and failing schools. Use the schools as support. Many are already failing at their primary task as first line instructors - just take it off their plate.
The current teaching model simply doesn't scale. It worked well in the past, when most people got like 4 years of elementary school, and a few selected ones studied longer. Now we want everyone to sit at the school desk as long as possible, and preferably in small classes. (Yes, I understand why small classes are better. I am just saying we can't afford them for everyone without compromising on teachers' quality.)
Just guessing the numbers: if people live 80 years on average, and spend maybe 15 years on average at school, that is like 20% of population being students. If you want 1 teacher per 20 students, that is 1% of population being teachers.
We all have an idea of what an "ideal teacher" should look like. And some of us actually met someone like that in real life.
But 1% of population having the required skills? Also, there are other professions competing for talent. If you make all good programmers teach computer science, who is going to write code? Similarly, who will do surgery, etc.
What currently happens is that rich people (plus a few lucky non-rich people) succeed to get the best teachers for their kids... and the rest have to deal with what is left for them.
With sources like Khan Academy, everyone on this planet could have a top-quality education. You only need to make the great video lecture once. It can even be translated to other languages by other people, who don't need to be good scientists or educators themselves, just good translators. Education could be free like, dunno, Linux or something.
And yes, having a good human teacher is better than watching video. I am just saying it is a luxury that most students don't get. But everyone could watch the great video. So maybe we could change the school to be more like... uhm, maybe watching the videos together at school, and then having a debate about them? Then doing the related online tests, and again having a debate about what went wrong?
We already accept written textbooks. We should do the same with videos.
I think that the point of textbooks is to lower the required subject matter expertise bar for teachers, leaving them to focus on teaching the material.
Unfortunately, when a teacher is merely familiar with the material rather than understanding it deeply, they will find it difficult to explore multiple (valid) ways of explaining it to students until it 'clicks' for each one, which means students often fall back on rote memorization and mechanical competence.
A video of a good (even great) teacher is no more interactive than a textbook, so if the student doesn't happen to 'get' the explanation as it is presented they are going to fall back on memorization and mechanical competence in exactly the same way.
There are interactive learning environments with extensive libraries of content that explore material in multiple ways as needed (as indicated via assessments) by students, but AFAIK they are all proprietary.
Without textbooks, it wouldn't be any better. Only the kids wouldn't have a secondary source of information.
I am completely in favor of having better teachers. I just don't see the way to find them, and people were already trying for decades, so whatever idea I would come up with, someone probably already tried it. (I know a few people who are working on a reform of mathematical education. It is from them that I know how the actual sitation is even worse than I imagined.)
> A video of a good (even great) teacher is no more interactive than a textbook
Agreed. Sometimes the video can contain helpful animations.
My hope is that the videos could be updated more flexibly than the books. When you print a book, the next version comes in a decade. There are known errors in textbooks, but no one is going to spend money to reprint them again, just to fix the errors. It we could have the books online, we could fix the errors as they are found... but most people are not going to read books online. However, with videos hosted online, I hope it would be possibly to, dunno, add the errata as subtitles, or something.
Another hope is greater fragmentation. A teacher is unlikely to recommend one chapter from one textbook, and another chapter from another textbook. Or recommend two textbooks, like "if you didn't understand the explanation in the first one, try the second one". With videos -- here I assume there would be a free version online -- this could be easier.
I have seen many videos recorded by teachers during COVID. Some of them quite ok, considering the authors are amateurs. But the thing I like most is that if you have five teachers that teach the same grade, and make the videos for the whole year, you get five alternative videos for each topic. Now when anyone says "I don't get it", you can easily give them four more links and maybe that will help.
Elementary school education is an area where you really need to think about how your solutions scale. If you do the right thing once, literally millions can benefit from it.
There's a surefire way to increase the supply of good teachers, increase salaries & benefits. Reduce classroom sizes. Give teachers adequate time to plan engaging lessons and make personal connections with students, their families, and the community. Teaching is a miserable job with low autonomy, high stress, a low pay ceiling, a rigid pay scale, and is absolutely draining.
>We can support much better outcomes for far more kids by reducing the dependance on the quality of your teacher, school and school district.
Sure, but the whole point is that this is much easier said than done.
Do you have time in the classroom?
This is the problem. There aren't enough good teachers. Salaries/benefits/class size/prep time are a function of funding. Low autonomy/high stress are the result of our regulatory demands that appear to be on a one way direction: even more requirements and testing.
One particular friend left high end consulting to become a HS math teacher in an inner city school. He came with an academic background - having taught for a bit in colleges. Very optimistic. He left after 4 years because the number one problem in his classroom was keeping the kids focused at all. As he put if he had three concepts to cover in 50 minutes, he was lucky if could get 1 covered for the 2 or 3 students among the 30 that were trying to pay attention.
He was a good teacher and paid well. The school, school district, the community had complex problems: low income, crime, food insecurity, parents who had to work multiple shifts, etc. He couldn't solve all these problems before explaining right triangles.
I don't believe the model that we have today will survive the next 25 years.
I agree that good teachers and good schools are necessary. But we don't seem to be generating or supporting enough of them. So I think the solution is to look elsewhere entirely. Break up the problem. Remove the dependence on good teachers and good schools.
BTW, if you're a teacher. Thank you for what you do.
>So I think the solution is to look elsewhere entirely. Break up the problem. Remove the dependence on good teachers and good schools.
I think this comes to our fundamental disagreement. I just don't think this is possible nor a good path to pursue.
Thanks for taking the time to engage in the back and forth!
> There's a surefire way to increase the supply of good teachers, increase salaries & benefits. Reduce classroom sizes.
I don't disagree, but either one of those solutions alone is a pretty hard sell, trying for both at once isn't going to get much traction.
Personally, I suggest focusing on classroom size first, as it should improve the performance of existing teachers and at least make the job less unattractive to new teachers.
This has all been done to very, very little effect.
Reading, Math and Science scores are basically flat since 1970. K-12 employee numbers have almost doubled over the same period and costs per student have almost tripled.
https://www.cato.org/blog/public-school-spending-officials-v...
Side note: This also increases demand for teachers, because any single teacher now teaches fewer students.
If you reduce class size by 1/3 you need 50% more teachers than before (at the same workload, to teach the same number of students). If you reduce class time by 1/3, you again need 50% more teachers.
Citation is needed here. I'm not buying that Khan Academy has better outcomes just because this is HN and the internet is cool.
Where is the evidence that Khan Academy is better?
Rant: Any one thinking it's easy to teach should try it sometime. I get especially fluffed up by would be reformers who've never taught. eg Gates Foundation. While I appreciate the interest, deploy all y'all whackadoodle ideas against your own kids before torturing the rest of us.
Source: Mom was a special ed teacher for 20 years. Incredible how many people knew so much more about how to do her job than herself.
That makes two of us.
Some of my students referred to it as The Power.
(I have no financial interest - just a customer.)
Get rid of government subsidies for education and let people choose what education they get, where and when. You'll quickly find plenty of willing, able and capable teachers.
While we're at it, get rid of minimum wage laws so that people who don't want to pay to learn, can do apprenticeship programs and instead get rewarded for their efforts.
I see a lot of people going crazy after YouTube content creators so I'm sure it's viable for education as well.
I would agree if you strengthened this to "for almost all kids..."
Effective primary education is greatly dependent on the bond between the students and the teachers (IMO). Yes, you can learn your times tables from an online lecture and online drill program, but you can't replace elementary school with it. Secondary education likely has less of this property, but still way, way above zero.
There is no singular "best" teacher of a subject. There might be one who is best for many learners, but I believe a large element of education is finding exactly the student's "frontier" and repeatedly crossing it, slowly expanding that frontier while spending enough time inside for reinforcement, confidence-building, and repeated checking that you're right about the frontier.
It's heavily labor intensive and personalized, which is why it's local and why it's school and teacher dependent.
Apparently another org will be hosting them through the end of 2021.
If you want to continue accessing these, check out the instructions about setting up Kolibri posted in the thread above, then in step 3 download this channel using the token "nojon-baraz" or simply select from list. Once you download to localhost (might take a while), you'll have all the videos and exercises locally so not a problem even if this materials are taken down from the KA website.
You can get the same channel locally (localhost; for offline use) using the following steps:
1. Download Kolibri https://learningequality.org/download/ , 2. Install and start, 3. Go to Device > Content > IMPORT and choose the channel called "Khan Academy - Standardized Test Preparation" from the list, 4. wait for import process to finish, then you're done.
I posted the same info in another comment on this thread of this story so I didn't want to repeat here. See that for more info.
"A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4."
Check the readme or requirements.txt
"You can download courses interactively on a prompt, which will list all course domains and their respective courses available with tab completion."
Instead of downloading from scratch, I would recommend using one of the pre-downloaded and pre-packaged Khan Academy video archives: a Kolibri channel or a Kiwix ZIM file. Details provided below:
## Kolibri
This is the best open source offline learning app developed by Learning Equality (the same people that made KA-Lite, see https://learningequality.org/ka-lite/map/)
1. Download Kolibri from https://learningequality.org/download/
2. Browse the list of Khan Academy channels to find the one you're interested in (there are 16 diffenret channels, for the different languages available from Khan Academy website) https://catalog.learningequality.org/#/public?keywords=khan
3. After installing and starting Kolibri (it's a web app that runs on localhost) go to DEVICE > CONTENT > and IMPORT the channel you selected in step 2. (Note download will take a while, like 8h+ because it will have to download 20k videos, exercises, and subtitles).
4. DONE! You have the entire Khan Academy collection in an easy-to-use fully offline learning management system. For bonus points you can create separate "coach" accounts as a parent and watch your kids progress.
## Kiwix
This is another powerful app developed initially for offline browsing of Wikipedia, but now extended to support many other sites. Kiwix is based on ZIM files, which are highly optimized compressed archives for web content.
1. Download Kiwix as app or desktop program https://www.kiwix.org/en/download/
2. Find the name of the Khan Academy ZIM file for your language and course, search for "khan-academy-" on this page https://wiki.kiwix.org/wiki/Content_in_all_languages
3. After installing Kiwix, import the right ZIM file (again it will take a while).
4. BAM! Now you have all the videos offline (exercises not supported in Kiwix, only the videos).
Between the two options, the Kolibri option would be my recommendation because you get all the exercises as well, not just the videos, but if you're looking just for videos, then Kiwix is faster.
- Learning Equality => https://learningequality.org/donate/ and https://github.com/learningequality
- Kiwix => https://www.kiwix.org/en/support/ and https://github.com/kiwix
- And of course the big papas https://www.khanacademy.org/donate who make all this great content available in the first place!
Khan Academy seems to have really raised the bar for this kind of content. I'm inclined to trust their decision on how to maintain quality and balance that with accessibility. Maybe I'm just naive?
At the same time, it does seem like having the content easily available offline could help people out. I wonder if they've ever explained their rationale?
Data wants to be free, as they say, and digital economies are a world of abundance where copying is free or nearly so. When a provider doesn't align with that they are imposing physical-world scarcity economics in a place where it doesn't belong.
It feels like people are trying to erase fundamental concepts like files or downloading, and instead offering a bullshit rental model where anything you can do on the net is at the pleasure of the content owner. It is disempowering to users as it changes them from masters of their domain to digital peons in CC's feudalistic kingdoms.
That is what I mean by 'greedy'. If you publish something online, it is out there for good. You cannot try to delete it and you cannot try to control it. You do not have a right to try to profit on the content itself because, again, data is free and no matter how much one might want to control data, ultimately they can't. DRM, ads, subscription gates -- these are all additional layers of crap that we have built to cage that data and prevent it from being freed.
It is a shame that we as a people cannot handle this world of abundance, and that the tools we use to navigate it grant us less and less control with every new DRM scheme or subscription service.
There are plenty of ways to monetize without locking down bits.
This is why I find it so disheartening that Khan Academy doesn't allow downloads natively. I understand the incentives for a provider like Youtube are overwhelmingly in favor of streaming-only, but an educational nonprofit like Khan Academy? If even they don't offer downloads, what hope is there?
Videos are one part of our work.
The articles and exercises are of course also important pieces, but there's no reason to make the video portion less accessible than it could be.
BOOKMARK that can help you download videos from many web sites