Yes. It looks like they introduce infinities and then run into the halting problem for infinities. That may not be helpful.
The place where this argument gets into trouble is where it says "we define hallucination in a formal world where all we care about is a computable ground truth function f on S." This demands a reliable, computable predicate for truth. That alone is probably not possible.
If, however, we are willing to accept a ground truth function with outputs
- True
- False
- Unknown
- Resource limit exceeded
that problem can be avoided. Now the goal is manageable - return True or False only when those results are valid, and try to reduce the fraction of useful queries for which Unknown and Resource Limit Exceeded are returned.The same problem comes up in program verification systems, and has been dealt with in the same way for decades. Sometimes, deciding if something is true is too much work.