Hacker News Reader

Comment by rahimnathwani | Hacker News Reader

I'm as excited about the potential as anyone. I would love to give my child an AI tutor like The Primer.

But nothing like that exists today, as far as I know. I was disappointed with Khanmigo when I tried it back in August. Although I have a year's subscription to it, I've not asked my child to try it. It seemed too boring and told me my answer (to a problem it had given me) was wrong, even though I was right.

I'm optimistic, but I think there's work to do. I've not seen an AI system that can apply the pedagogy of George Polya: give the student a small hint by asking a question that could have occured to the student themselves.

View on HN

> told me my answer (to a problem it had given me) was wrong, even though I was right.

This is a fatal flaw. It absolutely cannot make mistakes like this if it's going to succeed because that will undermine the confidence that teachers and students have in the system.

Here's a screenshot of the conversation: https://imgur.com/a/MYOQRK3

Aside from the wrong answer, I also didn't like the fact that it just told me to multiply the numbers, instead of giving me a smaller hint that might allow me to realize this for myself.

Amazing that it made such a basic mistake, so many times. I thought they wrapped the AI in enough layers of meta-cognition that it wouldn't make mistakes like this. Maybe AI in education really is like FSD, as another commenter said (so close, yet so far).

AIUI Khanmigo uses GPT-4 under the hood, so honestly I'm perplexed.

Khanmigo is a surprisingly naive product, but this isn't a particularly hard problem:

1. It's not necessary to invent all these problems from scratch. At least for any "normal" math problem the world doesn't need new problems on demand, you just have to fetch one. Khan Academy has a perfectly good bank of problems.

2. If you have a baked problem you should hang onto knowledge about the correct answer. Khanmigo just seems to put it in the chat history where it is easily forgotten.

3. If you want customized problems you might need to invent problems instead of reading problems from a bank. In that case you might want to create a more "formal" problem and an "informal" problem, where the formal problem is expressed clearly and solvable (e.g., by Wolfram or SageMath or something), and therefore can be verified. Translating the formal problem into a word problem is generally much more reliable than inventing a problem and correct solution.

That said, "right answers" aren't IMHO that interesting. Helping the student attack a problem in the right way, or adapt their knowledge to be able to apply it to a problem, is more interesting and helpful.

Here's an AI system that applies the pedagogy of George Polya: https://www.withmarble.io/

It helps learners understand, plan, and then implement the plan.

I really like the walkthrough video. If I were in your target market I'd definitely try it.

Reply on news.ycombinator.com