Hacker News Reader

Comment by stavros | Hacker News Reader

Of course it believes the proof is sound, it wrote it. If you want to check an LLM's output, you should use a different LLM.

View on HN

Your comment is not substantiated at all.

If you'd ever tried to get an LLM to review its own code, you'd know.

if you get the same session that wrote the code to review it the poor results are entirely deserved.

and if you get a different instance to review the code then you would know that it works rather well.

No, the comment is right. The prompt had GPT-5.6 reviewing the proof, and the result, unsurprisingly, survives review by GPT-5.6.

Given a new context, why couldn't the same model have a decent shot at reviewing some results? It's not like they identify whether this output is from them and then go "yeah correct", that's not how they work.

It’s the other way around. The prompt instructed a GPT-5.6 agent to try to make a proof that would survive review by a GPT-5.6 subagent. If there were some defect that would cause the reviewer subagent to accept an incorrect proof, then one might imagine that someone else asking the same model to review the same proof would give the same result. And the proof generation process might even be biased to find such an incorrect proof.

Use a human maybe.

Only people can really verify clankers.

Can't trust anything LLM since it will confidently lie too.

It can't take responsibility for verification so it can't verify.

Reply on news.ycombinator.com