By design, even the same LLM, when asked the same question multiple times, will almost never hallucinate in the same way. By that (flawed) logic, you could have the same LLM check itself.
Is it perfect? No. But it does what I asked it to do.
But when you ask the SAME LLM (with the same context) they remember the hallucination so it doesn't work. You have to use a fresh one without the same context