Why Sal Khan't: On Learning by Making but Teaching by Telling
punyamishra.com
punyamishra.com
> Now… how does Sal Khan want my kid to learn?
> Watch a video.
I find this a little uncharitable. I learned a lot of math at a young age from Sal Khan's early videos (back when he used to erase by switching to a black pen and scribbling across the entire screen) and, to put it in the author's terms, it served as an easy-to-digest scaffolding on top of which to build a deeper understanding. I don't think Khanacademy ever sold itself as a magic learning resource that removes the need to work out problems yourself.
I grew up in a small rural town where math teachers were rare and good ones were nonexistent. I still remember watching Sal Khan's videos on calculus after struggling to get any grip on it from my frustratingly uninterested teacher who was forced into the position after our actual teacher retired. After I saw them I came back with a fire in me because for the first time he exposed me to the beauty of calculus.
This guy is completely wrong about Khan not inspiring students to care. Sometimes all students need is to see what caring looks like to ignite it in themselves. You can feel how much he cares just by watching him because the video is the product of all that caring, and he does it for millions of people.
But thank you to this blog author for being brave enough to call out Khan's failures, I am sure it'll help education much more than Khan did.
No good deed goes unpunished.
Excellent point, well expressed.
(Also, IMHO the sarcasm in your final sentence detracts from your comment.)
They have some significant advantages compared to (just) a classroom: (1) They set a quality floor – sometimes a teacher can be really incomprehensible, or mean, or whatever, but Khan's videos are consistently okay. (2) They are available from anywhere, without need for special social/technical infrastructure, like enrollment in school or access to a university library. (3) They are self paced, which means they can better accommodate the various interest level, motivation, and speed of different students. (A typical classroom also has some advantages: it gives social setting, a consistent schedule/structure, and human feedback.)
I'm also disappointed though, because it seems like, given production resources, there would be potential to make widely available materials that were much better than average lectures. With significant production effort (a team of subject experts and content creators doing a lot of dedicated work on each topic) I think it would be possible to make something truly great, but Khan's videos fall so far short of my imagination. For any particular topic, there are typically other freely available resources which I think are better.
I've also been disappointed because many people have hyped the site as being extraordinarily amazing, but when I have looked at it, what is there doesn't match the hype.
Its better than average quality classroom teacher in my experience.
And the point is this didn’t exist online at the time. His resources were strictly more. I imagine its still hard to find a single free resource that matches its breadth and quality.
Do you have few examples?
There are? I know an OCW course includes "lecture notes" pdfs. Does it have the lectures too?
For example, the course "distributed systems" is available on OCW as 6.824, from spring 2006: https://ocw.mit.edu/courses/6-824-distributed-computer-syste...
MIT has renumbered the course, but it's still around: https://pdos.csail.mit.edu/6.824/ . Don't be fooled by the URL; it's now 6.5840.
And a series of lectures is available too: https://www.youtube.com/@6.824/videos . They're from spring 2020. They link to the course website for spring 2020, which is great. But it doesn't link to them, and OCW doesn't know about any of this, as far as I can tell. The 6.824 lecture series is pretty hard to discover, too, because it's uploaded by an account dedicated solely to that one course, which isn't related to any other MIT lecture videos.
I am having my kids learn math with Kahn academy if we don't have anything better by that point.
He sold his vision in "The One World Schoolhouse" which was essentially global learning via all the best videos (and one unified curriculum) you can make and teachers effectively being technicians/teaching assistants. It was the vision that utopian minded technologists craved in the early aughts.
"essentially global learning via all the Khan videos, in math, by him, and in other subjects, by his team."
Khan Academy is 2010-era state-of-the-art for rote learning and procedural knowledge. That's a helpful resource, but as one world schoolhouse, there is:
- No room for creativity, or much of anything similar from students (outside of the fantastic JavaScript projects).
- No room for outside innovation.
- A personality cult. Any ideas MUST be attributed to Khan.
It is the highest level of learning, but still a level of learning.
Only if your goal is to teach people how to solve that particular problem. But it’s not the best way to teach mathematics.
For my money, the best way is problem based. I heard of one math teacher who had a video playing as the students walked in. The video was a weirdly shaped bucket being slowly filled by a hose. Students came in, saw him sitting up the back, got fidgety then someone asked “how long will this take??”. At that, he shot up and said “good question”! And they spent the rest of the class trying to estimate how long it would take for the bucket to be filled. Complete with diversions into flow rate, volume, and so on. He made the students ask the question. Then got the students to try and answer it themselves, helping them work through their own ideas on the board.
That’s beautiful teaching.
Most people would disagree with you about the purpose of school.
Article cites Dewey, who wrote a lot of text about what school is for. Friere is also worth reading there. You'll see a diversity of perspectives.
"Now you add the five ... [pause] ... add the five ... add the five to the seven ... to the seven..."
Running them at 1.5 - 2x helps a bit, but it's still annoying to the point that I find it very distracting.
Similar with the trend to auto-cut pauses that speakers make, just to cram in more text into a youtube video. Destroys not just the pacing but any chance for the mind to take its time.
I'd love to see studies about this.
Playing videos at 1.5-2x fixes the problem entirely. My attention is held without any effort.
If a concept is new or difficult enough, I just slow the video back down or pause it whenever I need to think things through.
A good solution to this is to train focus. By giving in to the dopanine craving you are depriving your mind of the space it needs to process.
1. Teacher 1 speaks at 100 words per minute.
2. Teacher 2 speaks at 50 words per minute, for the exact same content as Teacher 1.
You're not really losing anything by playing Teacher 1 at normal speed vs Teacher 2 at 2x speed. For every learner, there will be an optimal teaching speed for a given topic. This may mean listening to some audio at normal speed, some at higher speeds, and some even at slower speeds. I always customise my playback speed to the specific content I'm trying to learn.
And I think the same is true in academic domains. If you're trying to think about something consciously, you're going to do a poor job of it. And the only way to get the unconscious competence is through a bunch of 'doing'. Chess is a great example of this because it's purely mental yet again you're never going to gain any degree of significant competence without playing a lot. Interestingly this even applies after mastery. A master who has played e.g. 1. e4 their entire life and then swaps to 1. d4 isn't going to be playing at the same level as he plays 1. e4 even if watches thousands of hours of videos on it beforehand. That connection you make by doing and interacting with things is just critical.
It is a spectrum. If you give your mind some space, it will start churning through the key ideas, fill in gaps, turn things upside down, ask questions. That starts during the explanations and their pauses. And ideally you have some empty time right after to think about it, maybe do an exercise, maybe try to explain it to somebody. Work it, like a good dough.
2x listening suffocates this process. Like too little sugar or too low temperature or too little rest for your yeast dough. It might "feel" good but you are missing out so much...
I haven't looked for the study, but I can if you are truly curious.
Though it's not the question I asked. Would the second group fair somewhere close to the first if it only had listened once?
(I don't buy the argument that this would not be fair since they would use the extea time to watch more stuff ... because that's rarely what people do with their extra time. Especially not watching the same thing again.)
Obviously one can overdo it. And there is variance in what people require. But not giving things time, instead giving in to the dopamine-craving, that's killing your learning abilities.
"Paris is the capital of France. Paris is the capital of France" -> not spaced repetition.
"Paris is the capital of France. [one day later]. What is the capital of France? [three days later] What is the capital of France?" -> spaced repetition.
And in fact, it's this plus practice where the real value lies in learning and memorising. The only purpose of an explanation from a teacher is to help you understand the concept. Generally once you understand it once, you don't need to understand it again - unless you forget, and that's where the practice / spaced repetition comes in. That's why I prefer the teaching part to be as efficient as possible, so I can focus my limited time on the parts that will make me internalise it.
I did the trial of Brilliant the other day. You can get 7 days free, then another 7 days added if you cancel the first trial before the end. After that you can continue to get 2 free lessons per day with ads. I think it's about $20 a month for unlimited lessons. The maths problems are more visual (which makes them quite intuitive) and it has a nicer, more polished UI than Khan Academy. It's easier to stay in the mood for doing more lessons. The main downsides compared to Khan are that the explanations are quite minimal (short descriptions, no videos), and it doesn't reach as advanced a level as Khan. I'm looking at it as more a way to do a refresher on the maths I already learned at school rather than a serious learning tool. If it were $5 a month I'd probably consider subbing for a couple of months and trying to blitz through all the material before switching back to Khan to fill in the gaps.
https://www.youtube.com/playlist?list=PL3B08AE665AB9002A
Personally I like it.
You also do not need to linearly follow the topics/videos. You can take a few quizzes from some topic, notice you don't understand something completely yet and instead pick up more trigonometry instead.
I've worked through Khan academy in a way similar to what the author describes as a good flow, but then all within the Khan Academy platform. The platform even allows for discussions and questions.
To me the problem is much more that motivation or a destination is not something that Khan Academy will give you. Then again, so won't most teachers. I much preferred Khan Academy over most of my math teachers, but occasionally I've had teachers who helped me love a subject I barely knew beforehand.
The other idea is that a student only advances to the next topic once they've mastered the current topic (e.g. by getting 4/5 questions right). The theory behind this is that if you don't understand a concept/topic then it is harder to understand topics that depend on that.
This does mean that students will be at different topics/stages. But because of the structure of the teaching a teacher isn't moving on to a new topic leaving students to fall further and further behind.
IDK. Maybe some sort of RAG-on-textbook solution will let us do it.
I have benefited a good deal from Khan Academy, and was quite disappointing to go back and see that they seem even more focused on showing videos than they did 10 years ago.
(I use the flipped classroom approach extensively. But the legal profession being the eager embracers of novelty that we are </sarcasm>, I refer to the approach as "enhanced Socratic method," given that Socratic method was something we all endured in law school.)
For math and engineering courses, the basic flipped-classroom approach was used at West Point for over 150 years: In the so-called Thayer method, students study textbooks in advance, then in class, they work on problems on chalkboards. The instructor moves from chalkboard to chalkboard, coaching the students in real time. [1]
(I do much the same thing in my contract-drafting classes: In small groups, students do drafting exercises using online whiteboards. I lurk on my laptop, quietly coaching in person as needed.)
[0] https://bokcenter.harvard.edu/flipped-classrooms
[1] https://flippedlearning.org/syndicated/avoiding-flipped-lear...
I remember minimal videos (3-13 min, usually 5-8), and then exercises drilling the techniques until you understand it and don't make mistakes. If you got a perfect score (no mistakes whatsoever), you went a little faster.
That's learning by doing, or am I wrong? Or are there no exercises for things other than math?
"What he has never had is pedagogical knowledge: an understanding of how people learn, what motivates them, what makes the difference between someone who pushes through difficulty and someone who types “IDK.”"
It is almost as though the author thinks that Khan Academy is an organization of 1, and that they don't employ learning scientists and have a content team and don't publish research papers in the learning science field.
Khan: Videos + rote exercises
Learning: Projects, complex problem-solving, exploration, argumentation, etc.
Dewey is worth reading.
Recall practice IS learning. Projects, exploration, and discovery-based "learning" sounds great in theory but in practice, without actual testing, really doesn't live up to the hype.
Adam Boxer is worth reading.
I was explaining the article, not my views, since the top-level comment did not read it.
Regarding your comment, it is as wrong as the article. BOTH are important. Recall practice is great for e.g. developing surface understanding, declarative knowledge, and well-done, procedural knowledge.
That type of knowledge is necessary, but not sufficient, for high-level performance in most fields, at least beyond a spelling bee or vocab exam. To understand gaps, see:
- Complex problem-solving, which includes extensive conditional, conceptual, metacognitive, etc. knowledge. (e.g. see how top participants prep for math olympiads)
- Attitudes towards school / learning (one of the top predictors of future success by 3rd grade, together with reading)
- Transfer learning
- Cross-cutting / not explicitly represented knowledge / literature on expert blindspot
- Etc.
"Projects, exploration, and discovery-based 'learning'" work great in practice, when properly structured, which usually means explicit, well-defined learning objectives.
Bad: "Make a nice poster on selling a car"
Good: "Independently explore an organ system of the human body. Make a nice poster synthesizing what you learned and present it to the classroom in three weeks."
If you're flailing around without having memorized basic trigonometric identities, you're going to struggle severely to apply them to classical mechanics. If you can't do basic matrix multiplication, you're not going to build a graphics engine.
The article tries to damn Khan with an example about probing how neural myelination works. If you understand the foundations of electricity (by learning about capacitance and impedance) and have done some grueling foundational learning in organic chemistry, ready answers might spring to your mind. You need the former to make asking the latter fruitful or meaningful.
Khan deliberately uses the term "scaffolding" - which since we're throwing around dead education theorists harks from Vygotsky - which is precisely what a teacher provides by pantomiming elementary concepts for a child. Ultimately the child needs to internalize these basics to succeed in more complex projects with a sense of independence.
I really don't understand this article. Two things can be true at once.
I agree with everything you said, except Vygotsky. :)
Vygotsky is worth reading, since:
- He is evidence-based. He isn't just a theorist. He cites studies. This contrasts with educational philosophers which dominated literature of the time.
- He is universally misquoted in US education literature, usually to say banally obvious things.
- He is brilliant.
- In the concept of ZPD, US education literature usually says the literal opposite of what Vygotsky said.
Do you really thing the concept of scaffolding didn't exist before 1900?
Curiously, Vygotsky precisely has a passage about (positive) experience with throwing special-needs students into the deep end of the pool on tasks they can't succeed with, and contradicts the (incorrect) intuition behind the teaching process implied by:
"Ultimately the child needs to internalize these basics to succeed in more complex projects with a sense of independence."
To be specific, a child does need to ultimately internalize these, but one of the best ways to do so is through a certain type of struggle, rather than breaking down into little pieces, in the ways consistently attributed to him.
In the classrooms I see through my kids, the teacher is doing roughly what a Khan video does: walking through a worked procedure at the front of the room. If a video fails to provoke the "wait, why does that work?" moment, then a teacher following the same flow likely fails in exactly the same way as the author claims and missing the opportunity for hands'on learning. A video actually allows the learner to go back and review content if anything was unclear, and presumably any interactive AI agent can help with the feedback part even though the teacher in the video isn't.
I also wish the piece had stayed with Khanmigo, where nobody has a good theory yet on where the ai tutoring loop actually breaks and why people fail to engage it and that putting it in their face may not work, why exactly personalization at that scale hasn't produced the gains it promised.
Also irked by the title built on a pun on Khan's name, sets up an awkward framing that kind of lets me know it may not be unbiased.
LLMs are great at this. I’m sure they’re misused constantly. But if you ask the right questions, LLMs are excellent learning aides. Especially for symbolic reasoning like mathematics and programming.
This article appears to focus on the chat bot which I have not used and was not part of my experience. At the time, I had a subscription to Wolfram Alpha that was helpful if I got stuck on an intermediary step.
The high level of goal of the post which is directed at asking good questions raises a point that seems valid to me.
[0] https://direct.mit.edu/books/book/5138/Teaching-MachinesThe-...
Also, 3B1B got his start at Khan Academy so that definitely counts for something.
But if we're in the context of classrooms, and discussions of how one might change the roles of teachers using computers, then I'm getting wary. Especially when a lot of money is involved from investors who do not care to study history because it might not fit their desired narrative.
Go visit a page like https://www.khanacademy.org/math/linear-algebra and count the number of:
* cookie banners
* donation panhandling modals
* account signup widgets
* gamification modals to tell you about leveling up
* etc
As anyone who’s met Sal will attest, this characterization is incredibly off the mark.
The author extrapolated from the product as opposed to understanding the person and how the product came to be the way it is.
> And this is what I think the edtech world keeps getting wrong. The assumption is that if you can deliver the right content, in the right way, at the right time, learning will follow. It won’t.
I think it depends on the goals of the student and what you're trying to learn. Some things are procedural and can be learned by being told, some things require a more hands-on approach to really "get."
If I want to learn how to drive a tractor, I just read the Farmer's Guide to Tractor Drivin' or find a YouTube video (I learned how to drive an excavator and fell trees like this—watch and listen and then go do what they said; hot damn, I learned how to do it).
If I want to learn how to produce a modern masterpiece (or novel invention) in some field or art, I won't find the "how to" in a book or video [1]. That's very much a personal journey that can't be replicated or recalled as a series of steps you tell someone to follow.
The Sal Khan way is prescription-driven and has its place in the world and I don't think it should be dismissed. But that way won't give you the "10,000 hours" level of understanding or experience (and arguably, shouldn't be expected to—even in the shape of an intermediary chatbot).
I grew up in India watching the early Sal Khan videos because I preferred them. They complemented school and helped me understand things I’d ignored in class or simply hadn’t been taught well.
You can critique the limits of videos without dismissing teachers who have genuinely helped millions of people learn.
For generic trope-y content like this, the voice can't be left this AI-ish. It has to be about something really interesting for this AI tone to be bearable.
The concept of Khanmigo was great. The implementation was basically useless.
It was a great attempt, but the article explains the problem:
"early OpenAI access"
This was literally the world's very first attempt at an agentic tutor, with (initially pre-release) GPT-4. That is worlds away from what's being built with fable or sol-grade models in 2026, or what will be available in 2028.
Educational technology that is focused on delivering “content” is unlikely to be of much value. Technology that induces the learner to experiment, to discover, to discern patterns, and to ask further questions, is going to be much more valuable.
Of course many Indian languages have aspirated and unaspirated "d" sounds, for example, and this is also reflected in the English dialet. But most of us English speakers can't hear the difference very well even if we concentrate.
I also don’t think he ever purports to be “all you need” to learn.
It reminds me of the people who criticize Duolingo because you can’t actually reach perfect fluency through their app--like wasn’t it always obvious that it was never meant to be the only education you need; wasn’t it always obvious that you need to actually _speak_ a language to learn it? And yet it’s still a great app that has helped millions of people learn languages.
I don‘t think Khan is solely to blame for this. These entrepreneurs would probably have done equal amount of damage without Khan, siphoning funds away from proven solutions like paying teachers and providing school lunches, and into ballooning administration costs and 3rd-party tech companies. However Khan is for better or for worse the face of this movement and will justly or unjustly get that blame.
Duolingo is however less ambiguous. There is very little language learning going on on their platform. You trick people into playing their stupid game and watching their stupid ads, thinking that they are learning, when in fact they are not.
(India is another story. If you are referring to ByJu, which was absolutely a predatory enterprise, I agree with you)
Note that I was explicitly not blaming this on Khan, I was merely stating that he is the face if this movement and as such will get a lot of the blame, regardless of how much he deserves it.
My personal feelings about him is that he is actually an excellent teacher, and has independently discovered what pedagogical scientists have known for decades. He however allowed his discoveries to be packaged as a neat capitalist solution, and then has watched as his “discoveries” were used to excuse austerity (and made a bunch of money doing so). At best he is naive, and at worst he is an active participant. Either way he deserves at least some of the blame (though personally, I would rather blame the system, and point to him as being a part of it)
However, true education is really hard. It requires hard work, sticking with something beyond when it gets boring and difficult. Being stubborn to get it right. These qualities are not very common and instilling this is extremely difficult from outside.
What author is asking is, he wants Khan Academy to be a true education platform , while dismissing it was video platform. He is wrong in both counts.
Khan Academy is a gateway for a student to draw the map or scaffold as the initial phase. The start of the education happens when the student works on the exercises, and tries it again and again to get it right. They are driven by the mastery based education which they try to ask the student to do.
A good general article on education, but weirdly wrong when it comes taking a dig at khan academy platform. And w.r.t to AI use in Education, students who know how to use AI are using it exceeding well for their true education.
Here's a presentation that, kind of claims the opposite. More specifically, it claims figuring it out on your own is not as good as having a teacher tell you, and then having you use the info, and it claims to be backed up by research
I'm reminded o A Young Lady's Illustrated Primer from The Diamond Age - and how LLMs haven't been able to achieve anything similar. With the correct supervising AI agents that have their own goals - I don't see why you couldn't have an LLM try to motivate you, or gamify education tailored to your interests and targeted at the areas you are struggling with. Somehow weaving in their own education videos seems way too difficult though - it feels like it should be an entirely different thing
Maybe Khanmigo just kinda sucks? Having a ton of funding doesn't mean its the best possible solution. I've never used it, so it's hard for me to judge.
I think there is a simple reason why AI tutors are not terribly successful.
AI is just not good enough yet, they are only just now capable of performing tasks, getting them to do those tasks involves a degree of careful guidance and the ability to handle frustration calmly and rationally.
Those things require a high motivation to have the task done.
They are not the properties you want in a tutor, a few students have the drive to learn in conditions that are not ideal. They are a small minority.
What you need is a AI that has a deep enough understanding of the ideas behind the metals it teaches and is engaging witty and charismatic enough to get that information across to a student.
When it has those abilities there will be a lot of parents that won't want it to have access to their kids.
Seeking help after a while is obviously fine but when the help is one keystroke away and always available, you take it before the useful part of being stuck has happened. Not ideal for learning tbh
It was never about replacing exercises, it was there to help you do the exercises from your actual coursework, and to serve as a refresher reference for things you might have already studied but forgot some details about.
Also the videos usually consisted of a while of time spent physically writing out and talking through examples.
I mostly used KA between 2010-2015, and received immense benefit. These were the years it was MOSTLY “just videos”. Having revisited more recently, I found the gamification, quizzes, etc. simply distracting.
My understanding is that he started KA as a private YouTube channel to tutor his own family members, made it public, and it grew in popularity organically.
I haven’t used khanmigo, but given the climate and evolution of LLMs over the last 6 years, it is a completely reasonable goal to attempt to employ them in learning (though it sounds like it was not as effective as planned for.. reasons).
KA never pretended to be “all you need” for learning. It was always at least an excellent supplement to a given curriculum, and Sal’s genuine interest in the videos he narrates always sounded (to me) as palpably genuine which also goes a long way. There is a reason most textbook chapters have an initial section called “motivation”. The primary job of a teacher is to incite interest, and potentially direction at strategic points…
As the adage goes: You can lead a horse to water, but you can’t make it drink.
Maybe we should focus only on having AI for the correcting/internalizing part instead? A purely deterministic grader and problem creator creates an endless stream of problems and solutions. The LLM's job is just to help you get to the point that you can assign a random problem from this problem set and the student will be able to answer it without assistance. Hopefully the teachers lecture at the beginning of class helped with understanding as well (I suspect that it would), and also provides a more broad "theory of mind" about what we are doing than the myopic LLM's "Let's get you solving this problem" does. Also, humans are social creatures, and being taught by another human being (even if it were less efficient) is likely beneficial to most people.
tldr; Normal human teacher, deterministic grading to gauge progress, and LLM assistants to help the student get to the point that they can pass the deterministic grading.
Think less science and more a conversation; establishing syntactic and semantic ground rules for the speakers grasp of a behavior relative to problem.
It's not necessarily about teaching something concrete. It's about teaching a social style in an abstract way. It's discussing a painting or piece of music, in math the symbols used to represent possible number/geometry generators.
I never just watch the videos. I rewrite what is written out. To recreate the song and dance so to speak.
Pragmatically speaking life is just performance art. Only a few specific realms of science move the needle for humanity; biological health maintenance (across all contexts; housing, food, not just medical) and our research in energy infrastructure.
Everything else is just one's relative outcome. Result of their sensory experience. Feels like this should be obvious at this point.
The studies that show brain activity goes down as we use AI. Yeah. So? Is it great to stimulate ourselves so much with ultimately arbitrary content our biology immediately tries to forget due to entropy? Heart attacks are up in <60 crowd relative to last generation. Constant interaction with flame wars and tabs v spaces (functional languages and Nix or gtfo!) computer config wars level inanity is not good Bob.
This resonates so much with me. I am the co-founder of a microschool and also a neuroscientist. And these two have intersected so much in fascinating ways. Four years in, the big lesson for us is that for learning to make sense it should have the following progression: meaning, motivation, mechanics, measurement.
Conventional schooling, and everything designed to help with conventional schooling like Khanmigo give in to this systemic gravitational pull of measurement (we must know what they are learning). And from that ensues your standardized curriculum, measuring the visible and easy, because that's easier than something like "mathematical thinking", and the focus on mechanics (multiplication tables, fractions, chemical names, whatever). And in doing this we leach out all meaning and focus on the drills.
Chatbots, the way they are set up now, are doomed to fail because the entry point is mechanics. "How do I solve this particular quadratic equation?" It's a meaningless question to a child because it simply does not connect to their world. We cannot start with that and work Socratically towards meaning. We must instead offer or build a world where meaning and sense-making are possible. (Wrote about this here https://blog.comini.in/p/why-khanmigo-will-fail )
Personally interesting for me is how meaning-making and sense-making are essentially the core of conscious experience. We, bundles of experiences, cohere and conspire and collaborate to create a meaningful experience from the sensory data that assaults us. Data that would be meaningless without a perspective.
So in education we are doing the same thing that the mind does so effortlessly by existing and being in the world and grappling with it. But we shoot ourselves in the foot by taking a reductive lens and split it into elements.
That's worked well for us only until we encountered complex adaptive systems that need us to also focus on the whole. But the system has been built to focus on the narrow and hope that these come together at some point. And so we end up with this meaning crisis we are all navigating, in school and outside.
More about Meaning before Mechanics here: https://blog.comini.in/p/schooling-has-a-meaning-crisis-para...
In the more core experience they were super focused on not giving kids answers. I suppose built on a fear of subverting the teachers or classroom environment. But again it became a fixation that took all attention away from teaching itself.
But I figure you put something out in the world and see what happens. Yet I came back to it a year later and it was exactly the same. Nothing had changed. It baffled me then, and still baffles me... I had done enough to know the whole experience was easy to manipulate and yet I couldn't see any evidence of effort. It felt very much like "we tried nothing and it didn't work". Maybe I was missing something, I don't know.
I think there's a few different problems here:
1. I think the author is correct to highlight that underlying motivation, which is so important to teaching. We can focus on a high-minded Constructivist ideal of intrinsic motivation, but motivating students is also what grades are for, and classrooms, and learning with peers, having someone pay attention to your work, and so on. I do think that the AI chatbot is not capable of providing that. But the AI can still ENGAGE with that motivation. I have a feeling Khanmigo never did. I suspect some of their privacy controls also kept it from ever creating a good model of the learner.
2. The entire experience was very focused on answers, on solving problems. And yet it focused on that so reluctantly. The obsession with not giving away the answers had the unfortunate result in not engaging with anything but answers. It felt like it was always teasing at something it would withhold. But questions are cheap, especially rote questions, they shouldn't be treated as precious.
3. This may be unfair, but I infer the makers of Khanmigo did not build it with love and craft, which is unfortunate. I don't know enough to say why. Though there's a weird confidence to the product and to all of Khan Academy that I feel does not bring the necessary humility.
4. The chat experience was asymmetric in the wrong direction. In each turn the AI can and will spew out paragraphs of text, and the student types a couple words. Chatbots DO NOT have to work like this, they operate wonderfully with large chunks of user input. But you have to have the modality to receive it, the openness of experience to elicit that response, and the respect to make use of that response. Speech can be good, but it's a real challenge in a classroom environment (though if you are investing in Khanmigo, you can also invest in the other hardware to make it possible).
5. In some fairness, individual tutors don't represent any pedagogy at all. It's usually just someone making it up as they go. Motivation is a huge aspect, studying with someone sitting next to you, watching you study, will do wonders for keeping you focused! But also a tutor will have a natural theory of mind, hopefully the ability to see a student get the wrong answer and get inside the student's head to model their understanding and find a path to a correct understanding. That's a very challenging task, not beyond GPT-4 entirely but still quite challenging, especially if you are optimizing cost or time to response.
6. A good experience will do a TON of prework. I don't know if Khanmigo does that, if they've reshaped and expanded each educational module into a thorough playbook for the AI. That's where you have a chance to model things like wrong answers (there are actually SEVERAL books enumerating wrong answers in math), and use that to plan out responses.
If Khanmigo is not successful, I think that's on Khanmigo. Still teaching is very challenging. And maybe all of Khan Academy fails in this way: it does not seem to acknowledge the challenge of what they're trying to attempt. They seem far too confident in what they do, and not nearly creative enough.
And the essay's premise is: this man wants your child to watch a video.