> But, color me skeptical

That is not a creative color.

> We will never solve the problem of a token prediction engine being able to generate a sequence of tokens that the vast majority of humans interpret as not corresponding to a true statement.

I think we already solved that problem by making sure the vast majority of humans never agree about anything.

It is true that we probably won’t ever get a machine trained on human output to ever be completely accurate (GIGO) but with the right systems and sensors we can at least get probabilistic accuracy. Let’s not forget how human consensus gets shaken up every so many centuries.