It isn't designed to know things. It doesn't know what exactly it knows, where it could check before answering. It generates an output, which isn't even the same thing every time. So this again is a problem of not understanding how it functions
If an entity can predict the correct answer to a question (with a sufficiently low margin of error), then it knows the answer.
However, if the prediction contains too much uncertainty, then the entity should not act like they know the answer.
The above is valid for humans and LLMs.
So we "just" need to model and train LLMs to take uncertainty into account when generating outputs. Easy, right? :)