Khan Academy and the Effectiveness of Science Videos | Hacker News Reader

Khan Academy and the Effectiveness of Science Videos

fnoschese.wordpress.com

72 pointstokenadult33 comments

I have experienced this first-hand. Let me give you a little anecdote:

I took Chemistry Honors as a sophomore in high school. Now a sophomore in college/senior in high school, I'm again taking Chem since it's required as part of my gen-ed. I basically never come to the packed lecture hall as I see it as a waste of my time to listen to a repeat of a class I took 2 years ago.

I print off class lectures and view the khan vids and it's pretty easy. Yet this hasn't really translated over to my performance in tests. I think a culprit may be that I gloss over bits when I feel this all has a "I know this stuff" feel.

Compare this to my biology honors course as a freshman in high school as well as again as a freshman in college/junior in high school again. I didn't know about khan academy then, and took notes in class for things that I wasn't sure about (it was all still essentially review --props to my great HS freshman Bio teachers). I also studied outside if class and did practice problems during my free hour right after class. Aced the class and more effort was put towards the learning. Any misconceptions/confusion was resolved with either a quick google search or question at the tutoring center for homework help at college.

I plan to try and see if the physics course I'll be taking soon will produce different results, since I haven't taken a physics class yet ever, so it'll be pretty much all new. I plan on first studying with mainly Khan Academy only and seeing the results, then with other more "traditional" methods. I'm mainly testing to see whether the newness factor has any effect on my ability to acquire and retain the content just from videos.

It's interesting that people are not really paying attention when the facts presented differ from their own mental models - I suspect this also happens a lot in school and university settings. I remember the only useful lessons were those that actively engaged the learners (and not in some phony interview style either, but real exchange with the teacher).

When watching almost any tutorial video, I find myself zoning out a lot. Often times I even open up the browser and read something completely different while the videos is still running.

A few weeks ago I had a discussion with a friend who challenged me to convince him his adamant assertion that videos are the inevitable future of all media is wrong. At the time I just felt instinctively video was never going to replace the written word and 2D graphical presentations, but I couldn't make a convincing argument. It only now occurs to me there might actually be scientific data in support of my suspicion and that instructions on video may have an information-to-brain bandwidth similar to a very boring teacher after all.

Unfortunately, most college lectures are even worse than videos. It is very difficult to not zone-out or become distracted in a poorly-lit room with uncomfortable chairs. And if you miss something, you are much less likely to ask a professor to repeat it than you are to jump to a previous location in a video.

I think the lectures need to be replaced by the videos as a home assignment. Classroom time should be split into two activities - intelligent discussion on the subject of the video, and actual problem solving with other classmates, with the teacher immediately accessible for any questions.

I teach at mathematics at a community college. I have videos for each section of the book and created problem sets for each section of the book. In essence, I've created my own book.

My intention was to invert the role of homework and classwork. That is, that homework would now consist of watching the lecture. Classwork would focus on solving problems. The outcome? Disaster.

I tracked usage of the videos. Around half the class didn't watch a given lecture. Around 1/3 consistently watched the lectures. The effect of this was that over time only around 1/3 of the class was able (and willing) to engage in problem solving and discussions about the material.

I had 2/3 of the students hopelessly lost by the tenth week. How does this compare to the traditional format? When I give the standard lecture/homework assignment type class around 2/3 will pass. I double the number of people passing but I strongly suspect that they end up failing in the next course. (It is quite easy to pass beginning math classes these days. ) My anecdotal experience is that while I get more people to pass by giving lectures each day I end up with a group of people who know the material less well.

My experience tells me that around 1/4 of the students will learn the material from my video lectures because they are motivated and willing to engage me when they are lost. Around 1/4 will stumble through the course and learn something regardless of the format it is given in. The rest are mostly hopeless. They lack motivation, drive, and the ability to work intellectually.

What you describe does work with motivated students and the right teacher. I am not the right teacher for this though. Without motivated students I don't think it works. That's the essence of the problem. Too many students don't want to learn.

In a traditional class a student can hide their ignorance. They don't raise their hand and don't participate. The facade of being studious and understanding the material can be maintained. When you assign lecture watching as the homework and then spend class time solving problems it becomes evident who is not keeping up. It can't be hidden. This causes negative reactions in my experience.

Here's a great lecture by Eric Mazur. It's long but well worth watching. It's called, "Confessions of a Converted Lecturer".

http://www.youtube.com/watch?v=WwslBPj8GgI

I wonder if shorter videos would help. I know when I did video lectures, it helped immensely to speed them up anywhere from 1.2x to 2x, depending on the lecturer and the audio quality of the video.

Additionally, watching the video could be an assignment that counts towards a grade. Sure they could sign in, turn on the video, and walk away, but this at least provides additional motivation to do the first two steps, which may be all you need to overcome the activation energy for many students.

Additionally, threats of failure help - I had a course that was considered the filter - about 50% failed every semester. I think I was motivated by that statistic because it ended up being just about the easiest class I had. The motivation was less about not wanting to fail and more about showing excellence where common people fail.

My videos tend to be under 20 minutes long. I try to keep them focused and to the point. No fluff. I agree that generally, shorter is better.

Motivating them to watch is a problem for me. I'm philosophically opposed to giving points for something a well trained monkey could do. I can't bring myself to give points merely for pressing play on a video. I'm slowly coming around though.

My company makes videos along these lines as well. We have found a sweet spot between 5-10 minutes per video (the shorter the better). Even breaking a long video into smaller chunks can help.

My experience with conference-style events, even virtual ones, is that they provide a sort of forcing function whereby you put time on your calendar and go and watch the thing. Saving a bookmark to watch some interesting looking video, webcast, or whatever at some future point "when I have time"? Doesn't happen very often.

So the problem with having videos that can be watched any time as opposed to going to class at a set time sounds like it could be a good idea. But I'm guessing it wouldn't work that well--except, as you suggest, with very motivated students.

Yes, I agree that this is a big part of the problem. I think that even passively engaged in a lecture during class imparts some information (perhaps by osmosis) and it's the forcing aspect of the traditional classroom lecture that is better than a video.

Overall I think the problem comes from a lack of a desire to learn. I'm not a motivator and so I think my students suffer from my inability to be a rah, rah type instructor. I don't know what can be done about it though.

The biggest problem is that the lectures are slooowww. It's almost impossible not to let your mind wander and start thinking about other stuff. The problem with going fast as a teacher is that not every student grasps things at the same speed, and different students grasp different things at different speeds. This is where videos shine; the teacher can go fast because the student can rewind if he missed something.

Essentially this is a critique of lectures and traditional teaching models which unfortunately uses Khan Academy as a pretext for discussing how lectures don't work very well.

The tiny feedback loop that Khan Academy quizzes provide just renders the whole point moot. And that's just one additional aspect of Khan Academy.

It's unfortunate because it's not as if the author doesn't understand that Khan Academy is much more than just videos -- it's stated at the start of his video. But all that is literally swept aside at the start of the video as it focuses on the same old stuff that we've known for all too long -- teaching via lecture doesn't work very well.

Anyone can put a video up on YouTube, but what makes Khan Academy fundamentally different is not just the quality of the lectures, but the structure of Khan Academy and supporting materials that go along with those lectures.

The irony is that the author seems to be reinforcing their own idea of video education by ignoring how Khan Academy doesn't fit within their preconceived concepts.

2 points:

1. Having quizzes post-video is still different than having a different teaching style. In terms of learning outcomes, the two styles could potentially be equal, however in terms of student motivation/engagement/interest, they are different.

2. I'm not disagreeing with you necessarily here, but I thought the quizzes are only for some math at the moment, and nothing else? This would mean that at the moment his argument applies to everything except some math concepts?

I should have said that I found the technique interesting and insightful. Learning doesn't happen without engagement -- that's the key point. I just thought it was a bit disingenuous to ignore what Khan Academy is beyond video lectures & "worry" about it being bad for students.

Re: Point 2. Yes, I just looked over the Khan Academy knowledge map and it does look like I was wrong; however, I'd still posit that there is more to Khan Academy than just videos (I've talked about the "Coach" elsewhere in this thread). But I can see the point the author is trying to make and I'm not asserting that the method is wrong, but that he's glossing over everything outside the video content of Khan Academy.

That said, I think the idea is a really good way to give students a better way to engage with the subject. It adds another way for a student to engage, which can never be a bad thing.

Yeah I agree that he did gloss over some of the other Khan features.

About the method the video talks about -- I bought some audio-only lessons for learning French once by Micheal Thomas that (for me anyway) confirms this method. On the audio, the teacher had 2 students present, and the lessons consisted of Michael teaching these two students. The teacher would often ask questions and test their knowledge, and the students often would make mistakes. Naturally he would always correct them. I found it incredibly engaging and effective for myself.

The article glosses over the most important part of Khan Academy: pairing the clear concise videos with practice/assessment. This is what engages the student "on a deep enough level" and reinforces the lesson. The only thing missing from the traditional classroom learning cycle is the possibility of something being explained in a different way if the student is still having difficulty.

> The only thing missing from the traditional classroom learning cycle is the possibility of something being explained in a different way if the student is still having difficulty.

I believe Khan's "Coaches" are for just this purpose. I really think that this is one of the more ingenious parts of Khan Academy. In traditional education, there was very little opportunity for collaborative learning because the roles of student and teacher are so well defined.

The role of a coach can engender and empower a collaborative learning environment.

I think YouTube replies would be great in this case, by others (and even Sal) using the same approach as Sal with Wacom and 10-15 min explanation of common misconceptions and corrections.

Sometimes I've wondered if the vision of Khan Academy executed to its fullest would be to have a Stack Overflow-like up-/down-voting system for user-submitted videos. Khan Academy is amazing (note: I'll be interning with them this summer, and I'm SUPER excited), but it's never going to be accessible for _everyone_ as long as it's just one teacher. I think Khan is a great teacher, but there are always going to be at least a couple people who are confused by wording in an unforeseen way. If everyone posts videos, and the best teachers' videos get pushed up to the top, then it's bound to round out the experience and help people who might not like Khan's personal style. This idea sounds good in theory (kind of), but it would never work in practice since videos are much more time-consuming to make and watch, and so could not easily be digested and voted on by the masses. Sure, there's the potential that if you open up the teaching to the masses that some really excellent material could rise to the top, but it would also be drowned out in the noise of countless sub-par teachers. With Khan producing all the videos, everything is guaranteed to be high quality.

Sidebar aside, the site does have a place for users to ask/answer questions below each video. What I realized while briefly mentioning Stack Overflow above is that it would be _amazing_ if that kind of rapid-answer community could be built up around these questions.

I've wanted to create such a site for many years now. My focus is on community college level mathematics. My goal is to make it topic based.

My college's administration is a bit conservative when it comes to new ideas. I think you are wrong about not having enough videos though. More and more college faculty are creating their own videos. What is lacking is a central storage location for these videos. I think people would be willing to upload their videos and make them publicly available.

I could, over time, what evolves is a book. In math this would work because at the beginning levels the knowledge is static. How to factor trinomials is not ever going to change. I think a Stackoverflow site would work for lecture videos.

Interesting idea. However, the voting metaphor is incorrect here, (as I think you even said). The only thing it really solves is that it determines what learning style is dominant.

If there existed an easy or common way of determining most favored learning style, then people with these alternative dominant learning styles could filter by those. I'd love that.

guaranteach is a video based teaching site, that matches videos to each student preferred style of learning. Here's a nice demo: http://www.guaranteach.com/community/exploring

Most probably missing so far, as to this is probably in Khan's roadmap or any determined volunteer.

Regarding "pseudoteaching" mentioned in the article - every great teacher is to some degree a con artist. They know that in the end it is up to the student to engage with the material - the only thing they can do as a teacher is to communicate their excitement about the subject and provide a point of view that would make the material easier to absorb, at least to some. Their goal is never to reach everyone yet everyone leaving their lectures feels like they have been made a party to some exciting secret.

I had a remarkably effective teacher who said as much, quite emphatically. I don't think most of the students actually believed him, writing it off as bluster. It was true, though, and this idea permeated his whole approach to teaching. It worked, too.

Years later, I still find this hilarious.

Khan Academy and other video teachers need to have lots of problems for each lesson for the students to solve. We learn best by doing.

For example, I remember my Calc II professor said that when he was in college, he'd do every single problem in the book for that lesson--even though only half of the problems were assigned (the ones w/o answers in the back). And look where he ended up: as an expert in calculus.

Also, I'm going through Stanford's CS193p (iOS dev) on iTunesU right now in my free time after work. Probably 80% of my learning happens by completing the assignments.

>Also, I'm going through Stanford's CS193p (iOS dev) on iTunesU right now in my free time after work. Probably 80% of my learning happens by completing the assignments.

Are these the assignments you used? Or are the ones on iTunesU different?

http://www.stanford.edu/class/cs193p/cgi-bin/drupal/download...

That page has the assignments, lecture notes, and source code from the lecture demos. The lecture videos are hosted on iTunesU.

I think if the students were told what they got wrong on the first exam, they would have easily done a lot better. Like some of the other commenters stated, constant assessment is key. Once people start to figure out that some of their most common conceptions are misconceptions, they'll start to question everything, and then the learning really begins.

The biggest flaw in tnis is that the OP criticizes khanacademy videos by showing that vastly inferior videos don't work. Khan videos are good b/c they make it easy to think: there are no distractions, just a blackboard and an intelligent voice "in your head" guiding you through the problem domain. The OP, instead of using a khan video or even a khan-like video, tries to teach physics using a video shot outdoors featuring the face of a talking girl with a guy juggling in the background. He might as well give his subjects a physics book and turn them loose in a sports bar to study while following ncaa basketball. Neither method is a good one for learning physics. What's sad is that thd OP is not just some crank on the internet, but apparently a newly minted PhD in education who will likely end up teaching teachers or running a school system.

Illuminating. Seems that the academy's recent focus on mastery and testing is of utmost importance, at least that way students who watch the videos but do not absorb the material will realize they don't actually "get it" when they can't answer the questions. The key will be in devising problem sets that truly require mastery and not just plug and chug or rote memorization.

One of my worst experiences in college is to have a lecturer run through some gnarly theory, after which you get the impression that you understand it.

Then he goes "whoops, I made an error just now", and everything has to be reassessed.

Interesting video. The theory the article demonstrates through its video is that people don't pay attention unless you begin with the popular misconception. Otherwise, people automatically believe they know what you're going to say.

I agree that it would help people with those misconceptions learn. But I think it would be a waste of time to listen to all possible wrongs if you are eager to learn and adequately knowledgeable about a particular subject. Maybe entertaining for the first few times, but inefficient in the long run.

Reply on news.ycombinator.com