It's a really hard problem to solve!

You might think you can train the AI to do it in the usual fashion, by training on examples of the AI calling out errors, and agreeing with facts, and if you do that—and if the AI gets smart enough—then that should work.

If. You. Do. That.

Which you can't, because humans also make mistakes. Inevitably, there will be facts in the 'falsehood' set—and vice versa. Accordingly, the AI will not learn to tell the truth. What it will learn instead is to tell you what you want to hear.

Which is... approximately what we're seeing, isn't it? Though maybe not for that exact reason.