Hacker News Reader

Comment by simonw | Hacker News Reader

What makes you think https://www.mathacademy.com/faq hadn't been evaluated by experts?

That appears to be their whole thing, and they've been in business for longer than LLMs have been around.

I think before that question is useful to ask, we have to know if that FAQ even says anything about LLM-based tutoring. After a few minutes of research, I can't find any evidence that Math Academy offers LLM-based tutoring.

This was linked from the homepage: https://www.mathacademy.com/how-our-ai-works

But more importantly if tptacek says they use LLMs and is a user of the platform that's good enough for me.

Everything described there sounds like old-school adaptive algorithms. I don't see anything about generative AI or LLMs.

I asked Google if MA does LLM tutoring and got back this answer:

> Math Academy does not offer Large Language Model (LLM) tutoring. While the company advertises itself as "AI-powered," this is in reference to a machine-learning-based adaptive learning system, not an interactive LLM tutor.

And here is a HN comment that indicates LLMs are a complement to MA, not part of it: https://news.ycombinator.com/item?id=43281240

You're right, I may have misinterpreted what tptacek said: he said he was using LLMs and that he was using Math Academy but I interpreted that as "Math Academy includes LLM features" - actually it's equally likely he's using Math Academy and having LLMs tutor him on the side.

(Confirmed I got this wrong: https://news.ycombinator.com/item?id=45439001)

I'm using LLMs alongside Math Academy. Math Academy uses machine learning generally (and so now they plug their "AI" technology) but it's not transformer-model-style AI ML; as I understand it, it's just driving their underlying spaced repetition system (which is interleaved through lots of different units).

In the scenario I'm discussing, Math Academy's content is a non-generative source of truth, against which I've benchmarked GPT5 and O4-mini.

You're confused. Math Academy isn't LLM-based. I use an LLM alongside it.

I think parent was clearly referring to LLM use, and not math academy.

Reply on news.ycombinator.com