Khan Academy Statistics videos are not good | Hacker News Reader

Khan Academy Statistics videos are not good

learnandteachstatistics.wordpress.com

39 pointsColinWright59 comments

I've seen more criticism of Khan Academy recently than praise. As someone who wasn't able to afford college and is working their way through MIT open courseware I find these videos helpful.

Sometimes I just can't wrap my head around one little concept and I research it until I do. Khan Academy and similar sites have been essential to me getting through multiple courses. Are these videos as thorough as a course taught by an experienced professor? Nope. But I don't think that's the point of them. It's a place where people like me can learn the foundations before moving on.

Before online courses existed people in my situation had little to nothing. Sometimes I bought used, out of date text books but when I got stuck on something that was the end. (This was in early 2000s).

Often I see people attempting to compare sites like KA to college courses. That's not really a fair comparison as I highly doubt that anyone with the option of college is choosing to watch KA videos instead. They are for people who don't have that option.

Because of sites Codecademy, Learn Code the Hard Way, Khan Academy, ect., I went from working in tech support and living under the poverty line to junior web developer. I will never claim that I'm as good as someone who attended professionally taught courses but I now have the ability to continue to learn and get better.

(This post isn't intended to be personal so I hope I don't cause any offence).

I often see people talking about how great they personally found a MOOC course, but that self-assessment is largely useless. To put it bluntly: you don't know what aspects of a topic you don't understand, were not told about, or whether the information you received was at all accurate, because you lack the expertise to determine that.

For example, people thought they had learned a lot from the Udacity Statistics 101 course and it received glowing reviews from participants. But unfortunately they did not learn about some of the most important basic ideas in statistics, and some of the things they did learn were just wrong:

http://www.angrymath.com/2012/09/udacity-statistics-101.html

the course is amazingly, shockingly awful. It is poorly structured; it evidences an almost complete lack of planning for the lectures; it routinely fails to properly define or use standard terms or notation; it necessitates occasional massive gaps where “magic” happens; and it results in nonstandard computations that would not be accepted in normal statistical work.

Astoundingly, the Udacity Introduction to Statistics course manages to go almost its entire length without ever mentioning or making any distinction between the population and sample in a study. I say I'm “astounded” because in my classes (and any one I've surveyed or looked at), this is the key idea in introductory inferential statistics. It's the very first thing that is mentioned in my class (or the book), and it's the very last thing on the last day, too. It's the entire reason why inferential statistics is necessary in the first place. In fact, the very word “statistics” means measures for one (sample) and not the other (population) – but you'll never learn that from this class.

"I often see people talking about how great they personally found a MOOC course, but that self-assessment is largely useless. To put it bluntly: you don't know what aspects of a topic you don't understand, were not told about, or whether the information you received was at all accurate, because you lack the expertise to determine that."

And? This also applies to to normal educational environments, not just MOOCS. Why do people assume that courses delivered on premises by an average teacher is going to be better than a MOOC delivered by world class guy. I'd wager that on average on premises course has more gaps/problems/mistakes than a MOOC. These on-premise courses will never be exposed to criticism, mistakes never seen. Moocs are exposed to every educational 'expert' there is, so lots of criticism pops up. That's a good thing. Normal courses don't have so many 'experts' looking at them.

You're dead on. Whether or not MOOCs reach the standards we want today, they allow far more scrutiny than the closed courses they'll end up replacing. Over time, this will help them improve, a great boon to everyone.

I like to think of Khan Academy, Coursera and Udacity as raising the lower bar for everyone. If your course / institution / whatever is worse, you can easily draw on them materials as a supplement. It you can find better, well, no harm done.

>Why do people assume that courses delivered on premises by an average teacher is going to be better than a MOOC delivered by world class guy.

Assume? I just gave you an example of a shit course delivered by a 'world class guy'. I've done plenty of real life classes in statistics and none of the teachers/professors were startlingly brilliant communicators but at least they were able get over the basic principles of the subject. They also didn't waste my time telling me how great they were.

You guys may be delivering good stuff, but there are a lot who are not. Remember how many community colleges/for-profits/universities/high-schools there are in the world. The majority of the teachers recruited were probably averagely talented. These courses will never be exposed to criticism, but moocs are. That's a good thing.

It's interesting that in a thread about inferential statistics you are asserting inferences based on anecdotes with very small sample size, rather than statistics. It's possible that there haven't been enough MOOC statistics classes to really make a significant inference, but that seems to suggest "we'll have to wait and see" rather than "they're all shit".

If you understand statistics you should understand the difference between anecdotal evidence and statistical data. You have one online course you're comparing to a handful of in-person courses you, personally, have taken.

There are hundreds of online courses and hundreds of thousands of in-person courses. You should know it's not a good idea to make inference about ALL MOOC's based on one example or ALL in-person courses based on a handful of classes.

If your statistics classes didn't teach you this concept, I'd say it was those classes that did a poor job.

Vague criticism which can be applied to just about anything in the world followed by some criticism of something totally irrelevant (udacity course) when the discussion is on KA videos.

I don't take it personally at all. I see your point and fully understand it. I consider someone to be an expert at something when they 'know what they don't know'.

But, what you are saying isn't limited to MOOCs.

When you are learning on your own to fully understand something, you don't let magic fly. When I learned my first programming language I didn't allow myself to use any libraries at all. I made myself make everything from scratch so I knew what every moving piece was doing.

In my opinion, the onus is on the student. I work with plenty of devs who have CS degrees from top schools who will look for a library to do something that can be implemented with a line or two or code if they would take a few minutes to learn about it. Sometimes it's laziness, sometimes they don't seem to have the ability to learn something without it being force fed to them. I've also had problems with some of them during code reviews because they don't know what they don't know, and in their case they have a piece of paper that says they are experts so they refuse to listen and learn.

These courses are far from perfect, but they are a stepping stone for people who have a burning desire to learn. It's 3:30am where I am. I've been working on an algorithms class since I got home from work at 6pm. (with occasional HN breaks). I'll get a few hours of sleep and go to work. This has been my life for almost a year. To say I lack any expertise at this point is a little silly. I have nothing but a HS diploma and I'm keeping up with MIT open courseware. I would say they are doing something right.

And I think that going from tech support to developer is a little more than a self-assessment.

It's a bit weird for a community college lecturer to be attacking the lectures from a Professor at Stanford, and then concluding that "MOOC's" are the problem.

What part of his critique doesn't apply to the real life Stanford courses taught by the same person?

Note that I'm quite happy to believe that famous people teaching at famous institutions are bad at teaching, for reasons he gets into in his final paragraphs.

I just don't see how that's a problem with MOOCs, except to the degree that people are choosing them based on prestige, and even then that's no different from the status quo.

Someone whose maths teaching videos aren't very popular complains that hers videos are better than the videos which are popular.

Khan academy isn't popular because he has money from Bill Gates, his videos were hugely popular on youtube long before that, for the simple reason that they satisfied user demand.

She needs to ask herself why there's more demand for people who want to learn about "mean, mode and median" than there is demand for learning about "levels of measurement and sampling methods". I'm guessing most kids and adults will never have to design randomized trials and data collection, but they will come across averages in every day life.

From her description she mentions how she spends a lot of time thinking about theory of teaching and rewriting and editing to correct mistake and edit out hesitations, what was noticeably missing was any discussion of applying user analytics, A/B testing, etc. to better understand how her users were interacting with her videos, when they were losing attention span, etc. One of the reason that Khan Academy is successful is they take a very metrics driven approach to iteration, something that should be very familiar to any startup founder.

Sure Khan's videos are flawed but people want to watch them. It doesn't matter how much better pedagogically Dr Nic's videos are if people don't want to watch them.

Satisfying short-term demand doesn't mean it's always the best option. For example, it might be in the best interest of a nation or people to plan for the long term and optimize for future demands.

Let's assume for arguments sakes that Dr Nic's videos results in better outcomes ten years down the line but no-one wants to watch them. What's the solution ? - you can't force people to watch Dr Nic's videos, and even if you could the outcome is likely to be significantly worse because you were forcing them.

Kids are forced to go to school. Are you suggesting they should not be forced?

Is this a joke? So the examples that she gives as to why KA is terrible are:

(1) He used a pie chart where perhaps a bar or column chart would have been more suitable.

(2) She's "pretty sure" he got the explanation of confidence intervals wrong, but was "so confused by the end" she can't be sure.

(3) The p-value video was accurate but dull.

And all this plus some other _very_ minor complaints ("he mispronounces the adjectival use of “arithmetic”, which is a bit embarrassing" - Seriously??) is enough to justify a massive tirade that makes accusations Sal is lazy, doesn't care, is incompetent, etc? I'm certainly more than willing to hear valid arguments KA is not perfect, but this is just rubbish from someone who frankly seems bitter nobody cares about her videos.

Bar charts are always preferable to pie charts. But the video was about "reading pie charts", so this is perhaps the one occasion when using a pie chart can actually be justified.

But if you're going to give instructions on reading a pie chart, the pie chart that you give as an example should at least make some sense in that format. Time-series data represented in a pie chart is abnormal.

It is really hard to substitute good teaching for kids K-12 with a video of someone drawing things on a board and explaining it. I work with kids all the time, and I see implementation of Khan Academy in US schools not really working. The only kids it seems to work for are those are personally motivated and are driven hard by parents/guardians to learn.

The MOOC model though does a lot to solve the access to education problem. If there are kids in under-developed countries who are driven to learn, and are able to access to MOOC videos then the potential impact is huge. I have a really good friend of mine implementing this strategy in low-income schools in Pakistan, and the results seem quite mixed. The kids who are driven will learn, whereas, the kids who need more attention and guidance will need a lot more teacher attention. Unfortunately, this seems to be the majority of kids. Teachers in the US also report similar results. Though for society in general it seems a net positive.

The other part of the problem is to engage kids to learn more. Engaging kids to learn is a much harder problem to solve, and possibly the most valuable one. Anyone in the industry who can provide a content network that truly engages and educates kids will gain the most.

On teaching: different people absorb and understand any educational material in different ways. The best teachers have strategies built-up, through experience, to make different kids understand the same concepts. The real technology breakthrough could be when we can pre-empt the strategy required to teach a concept to a particular kid, automate the solution and engage the kid by providing loads of cool ways to play around with the material through immersive experiences.

we have a mathematics professor how makes it that way. he once recorded the lecture and now we must look at the videos at home and we must make a cloze but we have to go to the university twice for 3 hours and here we work in groups and he only goes around and looks who needs specific help.

Before I was never good at Mathematics because i would always be stack at that one point and i couldn't ask anyone now i can.

btw: i dont think that moocs can fix the lack of motivation. This is a job for the hole society. Moocs are for people that want to learn something and can't afford it.

Khan describes his approach as "flipping the classroom" where kids would watch Khan's lectures at home or during lecture time at school, and then have a teacher who implemented activities and clarified any confusion with the material. I think it's a much preferable approach than one where kids are herded through concepts and tests.

And not only kids. I remember from my univ days (comp sci) we have this algorithm and data structures course. Typical lecture (2 hours) went like this:

-introduce some completely new (for most of the audience) algorithm or a data structure

-start doing advanced analysis of its complexity in various cases (or in case of data structure costs of various operations)

The problem with that is that when you encounter something new you usually need some time to get familiar with it, to play around with it for a while to get intuitive grasp of what that thing is and what properties it has. Without it it's very difficult to follow any advanced analysis because you probably don't even fully understand how the thing works for various inputs/doing various operations. As it was, 75% of the lecture was too difficult to follow for most of the audience which showed (most people zoned out and just mindlessly taking notes).

It would be much better if home assignment before the lecture was to read some theory and maybe try to implement the algorithm/data structure to see what problems you encounter and then you would come to the lecture with specific questions to answer and understanding what may constitute a hard part. This flipping approach makes a lot of sense to me. It's similar to learning by doing: you first get some very basic understanding and then try to do things and learn along. Traditional approach on the other hand is similar to reading several 500p+ books about OOP and Java and not writing a single line of code along the way.

This is another vague criticism mixed with some general platitudes. The platitudes are totally irrelevant in the context of people criticizing KA approach.

First, if someone is not motivated, be they adults or children they will not learn, especially in a medium like the internet where there is a lot more freedom in when and what to watch and practise. This has nothing to do with KA.

Second, good teaching can never be assumed just because it's done in a classroom setting. The major problem in education is getting good teachers. There is no system in any country where teachers are continuously evaluated and rewarded ( and get fired ) solely on the basis of performance. That is a big mistake. On the internet, real market forces work. ie people watch whichever course they want to watch. That's where KA shines through. He never forced a single kid to watch his short videos. But his viewership is now in the millions.

Agree myself. I think Khan Academy is very overrated as a whole.

If someone did a paper version that doesn't stutter through problems like Salman does and actually has some pre-planned structure and peer review, it'd be better.

Oh wait I just described the status quo before...

Your first paragraph may be right, but I think the rest is wrong.

Paper education materials have a recurring distribution cost, whereas digital materials have a larger up-front but mostly one-time cost. Investment in both digital education materials and widespread access to computers is what is revolutionary, not the specific lessons.

There is always some sort of blurb in an article like this that says something dismissive like "such a such a video which was proved bad by such and such people was switched out for a better one after the criticism". That's the point! One video file on some server somewhere is overwritten and everybody immediately has better materials than they did before. No new workbooks, no new editions of textbooks, no printing press or shipping involvement whatsoever, it's just fixed.

I'm sure a lot of these educators agree with that premise and would love to start a competing platform or work for Khan Academy itself on improving its material. Perhaps the real criticism is that Khan Academy is a major hype suck, making competitors hard to launch, and that it's apparent success has made it over-confident and closed to wider collaboration with educators.

There will always be some dis-gruntled teacher jealous of the popularity of KA's videos, and their criticism will generally amount to nit-picking some aspect or other.

I never seem to see any reasonable and valid criticism of KA's approach or his content. He himself goes to great length to state that his videos are not meant as a replacement to a classroom setting and yet some teachers with similar videos always bemoan KA's popularity.

I'm not comparing paper to interactive media.

I'd be happy with a regularly updated ebook, worksheets, Q&A stuff that I can do offline i.e. print it and sit in the garden/park for a bit with.

My main problem is that you can't consume video at your own rate, just the rate the author defines you should be consuming it, which in this case is "jumpy".

I've had the blessing of a good lecturer or two in the past. Salman is not one of them nor is anywhere near them.

I personally dislike video and prefer reading myself, I would however like to point out that video (compared with someone standing at the front of a room and lecturing to you) gives you much more control over pacing. You can pause, rewind, fast-forward and even, on Youtube and HTML5 video (though dependant on your browser), watch at various speeds faster and slower than real-time. (Khan apparently switched from tutoring his cousins face-to-face precisely because they said the preferred him on Youtube, which he'd originally done as a "second-best" offer when he wasn't available, as it put them back in control.)

Complaints from people who make videos and put them online (like the lady in the article) get far more respect from me than people who vaguely claim that you can't replace face-to-face lecturing, and that all face-to-face lecturing is of a high-quality. Doubly so since, I personally would prefer a well written article/book to either.

This particular lady's videos reminded me of suffering through bad powerpoints, that could be much better done as written material. All they seem to add is someone reading to me, and not in a particularly pleasant tone (think how much criticism she'd get from someone who does voiceovers professionally! These amateurs who don't know anything trying to do stuff themselves, not even realising what a fool they make of themselves!) At least Khan works through problems in the traditional blackboard manner, which seems slightly harder to translate to simple ebooks.

Ah, sorry, in a late-night stupor, I mis-read your "paper version" as literally paper version.

I totally agree with you personally, but I think videos are more useful for lots of people.

I don't believe watching Khan Academy videos as someone who knows the stuff well enough to teach is a fair game.

The value and merit of Khan Academy is that it does the job better for a huge amount of people, than anything before. Self-studying text-book devouring students aren't really included.

Though, maybe, there will be text versions of the video... Maybe a wiki?

I am hardly a big fan of Khan Academy, but not only is there hardly any precise or logically sound criticism in this article, the video presented as exemplary is horrible, just unbelievingly dull, the level of enthusiasm is about that of a voice synthesizer. Together with the strange aversion of mathematics for the sake of supposed "pedagogy", I find it rather unconvincing, to put it mildly...

Couldn't agree more. The video shown reminds me of the educational videos they used to show on the BBC in the early hours of the morning in the 90s so teachers could record them for classes... just mind numbing. I think we've learnt now that it's ok for education to be engaging.

I studied chemistry and physiology (anatomy) using Khan's video and I can say that it's not overrated at all, it's very useful way of explaining and it really helps you understand at least basic concepts.

Of course to learn anything you have to be motivated, either as an adult or kid. Otherwise you're better of doing it wrong.

I feel Khan Academy and current MOOCs need to be treated like alchemy or Freud.

It's people creating half-baked initial versions of a potentially revolutionary idea, even if it may not live up to the hype and never quite deliver on the promise of turning base metals into gold.

I don't know about freud, but alchemy did eventually turn into chemistry. You have to start somewhere.

I think that's the point he's making: these are early days and what we've got at the moment is likely to be partly or wholly wrong, but it's necessary and pointing in a good direction.

I agree. At least we should give them credits for making things change.

I'm still convinced that MOOCS are a scheme to keep research dollars in premier western universities.

Everything premier western universities do is a scheme to get more research dollars.

Not sure if that is a bad thing though.

It's not about volume, but concentration. We can't have adversarial countries attracting or retaining any talent, or at least a paying student body to utilize that talent.

as of today Khanacademy had 294,329,180 views on youtube. I think that is a little more than a "half-baked initial version".

I will not comments on learning statistics, since I not a expert.

But what kind of bothers me about these kind of blogpost that the are all really negative, as if Khan Academy was bad because some videos are bad. Even from videos that are not perfect about a subject people can still learn.

Im all for blogpost that start a discussion about improving some parts of Khan Academy, but the tone in this blogpost kind of bothers me.

Teaching statistics is harder than most people give it credit for being. One problem in teaching statistics is that many people teach statistics after not studying much statistics, as discussed in the article "Advice to Mathematics Teachers on Evaluating Introductory Statistics Textbooks,"

http://statland.org/MyPapers/MAAFIXED.PDF

which leads to generations of teachers and learners who think they know more about statistics than they actually do.

Another thing that is hard to teach about statistics is valid inferences from what kind of data were gathered in the first place, as discussed in the article "The Introductory Statistics Course: A Ptolemaic Curriculum?"

http://escholarship.org/uc/item/6hb3k0nz

It is easy to find examples of statistics courses and particular statistics lessons that are lousy. The harder task, as discussed by the authors I cite above, is building better statistics curricula for deeper understanding of the subject.

From the article kindly submitted here:

"I don’t like the Khan Academy videos about statistics. But I can see why some people do. Some are okay, though some are very bad. I’m rather sorry they exist though, as they perpetuate the idea of statistics as mathematics."

That's a very legitimate and cogent complaint, one worth taking to heart at Khan Academy.

I'd like to hear from HN readers who have taken up the article author's challenge of comparing videos about such topics as confidence intervals, which is a tricky concept I have to write about a lot for my work.

Why not: "Here's how Khan Academy Statistics videos could be improved"?

the Khan format doesn't work for everything and the spectrum is probably much too wide, but the appeal is exactly that it is one guy explaining writing. that way the students experience the thought process, which is what most teachers forget about (if they even know and care about the subject matter)

by the way, those videos on the site are terrible and the exact reason why this kind of "thing" does not and never will work.

If you are getting your education from only one source of information, then you are doing it wrong. Khan academy is just one source out of many on the web. I personally find their videos to be pretty superficial in content, but a good starting point nevertheless. If you really want to learn something in depth, read from as many sources as you can until the picture is clear.

I don't know why people point fingers at Khan Academy! I mean he is doing something good, something that no one thought of doing. Quoting from article- "Sal Khan made little YouTube videos to teach a family member maths. Other people watched them and found them useful. Bill Gates discovered them and threw money at them. Now there are heaps of videos, with some back up exercises, and some people think this is the best thing to happen to maths (and other) education" I think people are jealous because Salman Khan did something which other people might also do but they didn't! Even if they did, money might be the main motive where for Khan Academy, its the free & better education. Update: I was reading comments and found that lady saying "Unfortunately they take a lot of effort to make when you do it properly, and I don’t have the backing of Bill Gates." This is ridiculous.

"To start with the list of topics under the statistics heading showed a strong mathematics influence. This may reflect the state of the curriculum in the United States, but in no way reflects current understanding of how statistics is best understood."

Anyone producing teaching materials will tend to use the existing school or university syllabi as a guide. Search YouTube for a mathematical topic and include 'GCSE' in the search term to see a pretty good mapping of the range of teaching styles in UK/England and Wales (Scotland has a different system).

I think anyone here thinking of authoring materials for use by school age students should break the topics down as small as they can (high 'granularity'). That will allow teachers to include them in different programmes, and to shop around for different approaches.

> I’m rather sorry they exist though, as they perpetuate the idea of statistics as mathematics.

Wait, what?

See the articles linked from my comment elsewhere in this thread.

https://news.ycombinator.com/item?id=6302719

"I couldn’t find anything about variation, levels of measurement and sampling methods"

I'm not sure I understand how these are not all mathematical concepts. Anyone care to explain?

Khan Academy isn't good and never will be. It fills the gap for new college students who didn't pay attention in class and never learned how to read a textbook. True classroom education comes from sharing experience, not regurgitating material and doing practice problems.

Of course it is you who knows where "true classroom education" comes from. I can think of many actual examples of classroom education that happened without sharing experience, although maybe not "true classroom education".

What else is classroom education if you're not sharing experience? You can study on your own, so putting stuff on the board from a book doesn't count. You can do practice problems on your own, so going through examples from a book doesn't count. What else is left? The only thing to be gained from a classroom is experience that's unique to classmates and the instructor.

> What else is left?

Many people (including me) learn much better from listening to a person talk than from reading textbooks, even with no avenue to communicate back to that person.

You likely don't know how to read textbooks correctly. It's a legitimate problem in school systems around the world - most kids are taught from an early age to do exactly what the teacher says and that's how you learn. Thus, they never learn anything on their own.

The idea that different people learn in different ways isn't one I just came up with. See the brief survey on Wikipedia here: http://en.wikipedia.org/wiki/Learning_styles

I find the idea that my preference for listening over reading textbooks implies I don't know how to read them is somewhat bemusing.

I doubt you could come up with anything since you can only learn from other people, apparently. "Learning styles" is just another way of pronouncing "lazy".

Have you actually read about Khans theorys on how the class room and school should work? He wrote a book about it hand he gave mutlible talks.

Reply on news.ycombinator.com