>"outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept.

Sure, but that would require semantic mechanisms rather than statistical ones.