The word "lie" is probably too anthropic here. I should have just said "made up". There is no intent to lie. And the model isn't try to self-fact-check anyway. (Maybe some do). But if they do they are probably bad at it at the moment, at least from my experience of GPT3.5 (not used 4 much).

"Its a human nature to mislead others, sometimes knowingly." I read this line in an anthropology book. A similarly non-cynical approach towards your fellow is "trust but verify".