how would you prove that a human has it?
LLMs have static weights, therefore they cannot not have a concept of truth. If the world changes, they insist on the information that was in their training data. There is nothing that forces an LLM to follow reality.
ETA:
To elaborate a bit: based on your response, it seems like you don't think my question is a valid one.
If you don't think it's a valid question, I'm curious to know why not.
If you do think it's a valid question, I'm curious to know your answer.
Why not? We have optimised for truth and we are predicting the best words that ensure this optimal value.