You're right of course, LLMs provide a partial, unsound oracle.

The "halting problem is unsolvable" argument relies on the oracle not being able to output "not sure". But adding that option admits trivial oracles, like ones which output "not sure" for everything, so some are better than others.

The "real world" use most people have for halting oracles is as part of software safety, where if the checker outputs "not sure" you modify the software until the checker can decide if it halts.