I'm impressed that you give precision and recall metrics for this -- and amazed that they are non-terrible. I'm amazed because a fully general hallucination detector is obviously a truth oracle -- it can answer any question about anything in the world, by framing the question as a statement and then asking whether that statement is a hallucination.

From among the analyses the tool makes, it makes sense to me that contradictions can be detected, since that doesn't require knowledge of the real world. I'm very interested in how you do this detection ("Logical inconsistencies") in practice. Likewise for "Logical progression".

Two questions:

1. Since "overconfidence" is treated as a red flag, won't applying your tool as a filter cause LLM response precision to drop, often unnecessarily? The safest answer an LLM can give to "When was the Eiffel Tower built?" is surely along the lines of "The Eiffel Tower may or may not have been built at some time in the past."

2. I don't see how this tool can detect the kind of hallucination that (a) involves no contradiction and (b) requires knowledge of the world. These come up often. Examples: Citing plausible-sounding but nonexistent court cases, calling plausible-sounding but nonexistent methods in an API.