LLM or transformers just merely extracting signals from human text and build a "contextualized" predictor over a long sequence of words sorted by the information (technically it is attention) of each token, then generate sentences that way, one by one into other sequences at a time.

But the biggest problem is, even human itself is subjectable to hallucination. That is called being delusional, or being drugged. So it is inevitable from the first principle.