Hacker News Reader

Comment by quietbritishjim | Hacker News Reader

They are definitely doing more than parroting. When learning mathematics, it's not uncommon to have some little misunderstanding that just isn't covered by a written explanation because it's "obvious" unless you happen to have misunderstood it that particular way. When that happens, you could look up the topic in 5 books on the subject and not make progress (never mind that you have to search through them to get to roughly the right point). LLMs are genuinely good at challenges like this - they make mistakes, but they can still identify and correct your misunderstanding with a bit of back and forth.

A good human on the subject will do better, and I'm not claiming the LLM has "understanding" in any philosophical sense. But it's more useful than something that just regurgitates material.

View on HN

> it's not uncommon to have some little misunderstanding that just isn't covered by a written explanation because it's "obvious" unless you happen to have misunderstood it that particular way... LLMs are genuinely good at challenges like this

For undergrad math, sure, and again, I think it's often because on some random corner of the internet someone else had a very similar misunderstanding. On the other hand, when I ask for clarifications on niche topics that have few examples on the internet LLMs (even Opus 5) often confidently cite irrelevant papers / results or simply hallucinate.

I think you're desperate to think that LLMs cannot make any sort of deduction by themselves: they must have just remembered someone else doing it. But I've definitely had conversations where they've had insights that can't be explained by that. Yes, I've had plenty of hallucinations too, but it's worth it, for me at least, overall.

Having said that, I wouldn't attempt to learn a whole topic with an LLM, that would be frustrating.

I am not desperate; I just have enough expertise and work in a niche-enough field to recognize that LLMs just are bullshit machines.

See https://claude.ai/share/6431ccd9-f4f8-4052-8996-b046d1ea1f76 for an example that I ran into just now.

We all run into hallucinations like in your example, but that doesn't mean that the AI cannot make connections between different areas of knowledge it has, and combine them usefully to something new that wasn't there before. Like Navier Stokes, for example. I've had it make plenty of such connections in my own research, giving me new results I didn't have before.

Reply on news.ycombinator.com