Hacker News Reader

Comment by salmankhan | Hacker News Reader

2 million unique students are viewing our videos every month by choice (as opposed to being forced to attend class by the state). Read any of the thousands comments on our YouTube or Facebook page and the underlying feedback from students is that they are finally getting the "why" at the Khan Academy as opposed to the "memorization-paint-by-numbers-what" in their classrooms. Do we think we're ideal? No, we're constantly using data and feedback to iterate on our content (and unlike traditional lectures and textbooks, we actually have data on usage and comprehension).

If you truly believe you have a far better way to teach physics, you really should let the world see how you do it. Make Youtube videos and point us to them. We're looking for other great teachers that are consistent with our mission and resonate with students.

View on HN

Hi Sal,

Given some of the criticism of your YouTube videos, I agree with your simple advice: Let them make something better and let the viewers decide.

The Khan Academy does not have some kind of monopoly on providing education content to help people of any age learn. Rather, the Khan Academy was simply the first to resonate with so many people around the world in such a short period of time. You clearly have many people's attention.

The Internet is an open medium to provide content, and to those who think they can do better, they know that they don't need much to get something up, as has been made very clear with Sal and his use of a simple drawing tablet, screen recorder, and headset. So make it happen -- show how your content is vastly superior to Sal's that both educates and makes it fun. Oh yeah, and free.

Sal, as you have noted a few times now, you are actively looking for other people that posses your talents to educate people. Have you thought about creating an open competition on your site and let the public vote on what they think is good content and fits the "Khan Academy mold"? Just a thought :-).

Hi Sal,

I think torque2's point is that the way they teach physics cannot be encapsulated in a YouTube video. Physics teaching based on the last 20-years of PER requires engagement of prior conceptions and inquiry-based activities (to name a few key findings). This cannot be done in the type of teacher-centric videos you're giving. I'll admit that your lectures are awesome... but they're based on a learning traditional that has been proven to be ineffective relative to student-centric environments by a couple decades of education research.

I had a serious question that I hope you'll answer: "When you learn the physics required to make a video, do you just watch MIT lectures or do you go out and actively search for and build your own understanding of the concepts? Do you talk to other people about the ideas you have and what they think about the subject? In short, do you have a conversation with the material, or is the material hand-fed to you from a video?"

Thanks for all your hard work Sal, it definitely has a place in a learner's experience... it just might not be the place it's currently being touted as in the media.

And here is the problem: YOU ARE POPULAR.

And because something is popular doesn't mean it's good. Take modern music. A lot of popular singers attract crowds of people and make tons of money. But very often the music just sucks. And people forget these pop idols as fast as they make new ones.

Let me guess, smarty guys like just like you should dictate curricula and taste?

"2 million unique students are viewing our videos every month by choice"

Page hits isn't a very good metric if you're trying to get feedback about what viewers actually thought of the videos. A lot of people have heard of Khan Academy through the news or may land there via a search engine. How many of them watch the video all the way through? How many stick around and watch parts II, III, etc.? I could be totally wrong about this (please correct me if that is the case), but my suspicion is that YouTube's stats probably aren't sufficiently granular to distinguish between a page visit in which the viewer watches the video in its entirety (suggesting that the video was perceived as worthwhile) and a visit in which the viewer watches for a little while and then bails (indicating that it was not perceived as a valuable experience). User comments are subject to a great deal of self-selection bias. Viewers who thought the video was the best thing since sliced bread are way more likely to take the time to login (if already registered) or register and login (if not already a registered YouTuber) than those who thought it was a waste of time.

There's also a more fundamentally important question to consider: Did the viewer actually learn the material or were they simply duped into a false belief in their own competency? HN regularly features posts about interviewing techniques aimed at sorting out real programmers who can write code proficiently from people who simply possess a false belief that they are programmers. Sound pedagogical methods require some means of verifying that the student has, in fact, built a valid mental model of the subject matter and, of particular importance in the sciences, is capable of applying their knowledge to reason about novel situations. How do you know that your students actually understand the material? Do you have any way of distinguishing between students who have had a previous formal learning experience with the subject matter and are simply looking for a refresher course, students who are concurrently receiving other instruction (the teacher is using Khan Academy as a supplement), and students who have had no prior exposure to the material?

Wow. Here comes the bury brigade to crush my dissenting opinion without bothering to address anything that I said... How dare I criticize the sacred cow? How dare I challenge conventional wisdom? Conform, conform, conform. Everybody loves Khan Academy.

Well if you want to pay great teachers to produce science videos, I would start by offering this guy a lot of money.

Or at the very least, please watch the whole video because it explains one massive problem with video science lessons and how to partially fix the problem.

http://youtu.be/eVtCO84MDj8

He shows someone video not produced by us. Then does a small interview of a handful of people who watched those non-Khan Academy videos. Then he uses their lack of understanding to impugn Khan Academy. The person who made this video clearly cares more about their agenda than real science.

The irony is also thick with him using a video to try to explain that videos can't explain things.

My goodness, I think you are missing the point entirely. Or perhaps you mean to. If you watch the video, he says that videos CAN be effective. You sound so defensive here, it's starting to make me doubt your sincerity.

Wait ... what? Did you even watch the whole video?

First of all, he says all kinds of positive things about the work you're doing. "Impugn Khan Academy"? Really?

Second, he is explaining physics education research that he himself did for his Ph.D thesis. He's using examples from his own work to make comparisons in technique to what worked and what didn't work in his own, actually objectively tested research. This is not "a small interview of a handful of people".

Third, he's not saying videos can't explain things - he's saying that they often fail to make students question their own pre-existing misconceptions.

I hope you actually watch the whole video and catch the research context? I mean I am handing you a research-tested science education technique that you could easily incorporate into your own videos if you actually wanted.

Haha-- clearly this is an example of the ineffectiveness of videos. Khan's preconceived ideas have prevented him from learning what was presented!

On a more serious note, however, as a science educator, I appreciate the Khan Academy's work to make these very clear educational resources available to all. They definitely can be useful in many ways. Now if only my district would un-block YouTube...

However, I have to agree with many points Frank and others bring up. I have taught for 5 years--my first 4, I worked relentlessly to provide thorough and clear explanations for all the science concepts I taught. And I was baffled when I graded my final exams and found that my students learned little to nothing from 9 months of clear explanations and practice. Last summer, I learned about Modeling Physics (http://modeling.asu.edu/) and implemented it in my physics and chemistry classes this year. Now, my students have rocked their final exams and shown huge improvements from pre- to post-test on standardized instruments. This is simply my experience, but there really is significant physics education research which points to the same conclusions.

Educators are not trying to belittle or denigrate anyone. However, we have to protest when so many resources and media attention are being given to a method of instruction that has been shown to be rather ineffective, particularly in science, when incredible alternatives are available but are, unfortunately, largely being ignored and unfunded.

Working for the Khan Academy, I've seen the link to this video a bunch of times. Naturally, I was really curious about the voracity of the claims, so I did some research. It's actually worth reading the thesis as it's pretty interesting stuff. It's here: http://www.physics.usyd.edu.au/pdfs/research/super/PhD(Mulle...

The study was, as best I can tell after reading through much of it (a lot not relevant to this discussion), quite limited. The largest number of students considered for any of the experiments: 370. All from one school. Most of the other experiments included far fewer students (fewer than 100).

Chapter 9 (additional experiments in chapter 10), and the section on Participants and Design has most of the relevant information. I was particularly fascinated by the decision to remove students from the final result for "watching more than one multimedia treatment." I mean I understand why it's important to control for this, but part of the point of KA is to provide additional treatments of information.

I am not saying that this in any way invalidates the research, but I also don't think it's fair to say that this is somehow a surefire tested solution that is being handed over. In addition, because the research hasn't been repeated outside of the original setting -- at least I couldn't find it, please correct me here if I'm wrong -- it makes the claim that "because introducing misconceptions was effective in this study means it must be effective in all science videos" a pretty serious stretch.

I'm all for research. We're doing a lot of it right now with our pilot students/classrooms and existing users. But just because something has been research-tested doesn't mean it's automatically the right thing to implement for KA or its students.

Definitely interesting insight regarding how people can hold a particular perception and how those perceptions need to be addressed. And although the focus is on science, it can equally be applied to any subject: English, history, math, etc.

With respect to the Khan Academy, it would be interesting to see tools that provide a kind of interactive exchange of preconceived ideas and then helping to break them down. Having tests after the exchange would then help provide great metrics on the retention and understanding of the subject.

In any case, 1veritasium has at least provided an thoughtful response to the Khan Academy. He recognizes a good thing but provide some nice constructive criticism that could certainly improve what the Khan Academy already could offer. So as someone who enjoys Sal's videos, this idea gets a +1 from me... You know, as just some random person on the Internet ;-).

Hi Sal,

The whole point is not who can make better video lectures. The point is why lecture at all?

I wish the world would see my YouTube video about Modeling Physics (and resources) here: http://bit.ly/ModelingPhysics

I wish the world would see my 13 other videos showing Modeling Physics in action: http://vimeo.com/channels/modelingphysics

I wish the world would read my post on pseudoteaching: http://bit.ly/MITpseudoteaching

I wish the world would read my posts on standards-based grading, which allows my students to show growth and mastery in physics: http://bit.ly/SBGposts

One of my pseudoteaching posts talks about how forward thinking universities like MIT (your alma mater) are switching from lecture-based physics to a more interactive model like Modeling. The research on Modeling is there, please don't ignore it: http://modeling.asu.edu/modeling/Mod_Instr-effective.htm

More research --> The 3 key principles from "How Students Learn: History, Mathematics, and Science in the Classroom" are:

#1. Engaging Prior Understandings - "If students’ preconceptions are not addressed directly, they often memorize content (e.g., formulas in physics), yet still use their experience-based preconceptions to act in the world.

#2. The Essential Role of Factual Knowledge and Conceptual Frameworks in Understanding - "What novices see as separate pieces of information, experts see as organized sets of ideas."

#3. The Importance of Self-Monitoring - "Appropriate kinds of self-monitoring and reflection have been demonstrated to support learning with understanding in a variety of areas. In one study,15 for example, students who were directed to engage in self-explanation as they solved mathematics problems developed deeper conceptual understanding than did students who solved those same problems but did not engage in self-explanation."

You can read the entire book for free: http://www.nap.edu/catalog.php?record_id=10126

Modeling Physics does all 3. Your videos are 0 for 3. Badges do not count as "self-monitoring" in any meaningful sense.

You also have no hard data (as of yet) as to the effectiveness of your videos. Just anecdotal comments from students. You have no controlled studies. In fact, controlled studies show that video lectures are rather ineffective. See Derek Muller's video abstract and research: http://bit.ly/KhanEffectiveness

I'm not trying to be mean. I think your videos can be a resource for some teachers/students. But using video as the primary method for content delivery is ineffective.

We should not have to "flip" lectures and HW in order for students to be more interactive in class. When inquiry is done right, the interaction is already built in.

I'll stop there, before this gets too long.

Thanks for your time, Frank Noschese (aka torque2)

People aren't going to see your videos because you have a bad attitude.

In fact, controlled studies show that video lectures are rather ineffective.

I would assume that studies comparing a competent physics teacher to a video instructor would show the former being more effective.

However, there is a shortage of competent physics instructors. There is a (possibly bigger) shortage of competent maths teachers. I am speaking from my more extreme experience in South Africa, where most of the teachers are abysmal. Expensive video-based instruction helped me achieve a respectable mark in physics/chemistry, most of my peers lacked that privilege and did very poorly. Even at university, things like differential equations, linear algebra and calculus were not particularly well taught - most professors are not really good at teaching, and in my case, they were teaching in a second language (Afrikaans speakers teaching English speaking students). The Khan Academy would have helped me even at the undergraduate level.

I have posted to HN in a related thread about video instruction in South Africa [ http://news.ycombinator.com/item?id=2351100 ]. People have been doing it for years here. I would recommend a controlled study in South Africa using KA, or The Learning Channel videos by William Smith (NOT run in an elite private school, but at an average township school) before we write off video based instruction as "ineffective".

The Khan Academy has an interesting model, not necessarily for the videos themselves (which as far as I have seen are basically a modern version of videotaping a college lecture, albeit a well-explained lecture by an intelligent teacher) but more for the infrastructure built up to support the videos. I won't go into my views on lectures or flipping the classroom - others have already voiced my concerns on that front.

One item in particular that hasn't been much discussed here is the self-paced exercises. In here, students can try a problem, to see if they understand the content; and they keep trying until they get 10 in a row. The Khan Academy logs data about how many tries and how long it takes, so they have data on what they consider to be "mastery" (this may be an oversimplification...) There is also a certain game-play aspect which I think is kinda cool, and I like the map of linked concepts/exercise sets. As far as I see, they only have math problems up there as of now.

At face value, it looks like plug-and-chug, drill-and-kill methodology that irks any teacher of merit. But: there are opportunities to take this infrastructure and make it so much stronger ("formative" vs "summative").

Diagnoser.com is an example of an online exercise bank that includes aspects of several decades of physics education research (also some chem and life science) but the concept could be generalized for other disciplines as well. Each question includes "distractors" (wrong answers) that appeal to certain common misconceptions (what they call "facets" of understanding). If you get it right, great! you go on to another problem that might pose a similar question with a twist to challenge you. If you get it wrong, it recognizes from your answer what your misconception might be, gives you a quick mini-lesson to challenge your thinking on that, and gives you a related (though not the same) problem to see if you get it (for physics teachers: like an interactive FCI). The data is reported to the teacher, including right/wrong answers, and the "facets" that correspond to certain ideas and misconceptions. The teacher can see where their teaching (or in this case, Khan videos) have been effective, and where they could use more work (perhaps suggesting the student watch or review a particular video). NOTE: this is only one aspect of the Diagnoser Project's larger program, so I have taken it out of context, but it seems appropriate for this discussion.

This formative approach is effective especially for the conceptual problems, not just the plug-and-chug problems that I saw on the KhanAcademy.org site. Yes, this is much more difficult to program than "right/wrong, next!" problems, and requires research to write good facet-based "distractors" (take advantage of years or decades of education research out there!) but it (a) gets more useful data than just "right/wrong" and (b) allows students to act on their wrong answers immediately by challenging them to think about why it was wrong. I think incorporating something like this with Khan's gameplay/scoring/data collection methods would make the exercise feature much stronger.

Reply on news.ycombinator.com