It would be helpful to tell users that it's just a model producing mathematically probable tokens but that would go against the AI marketing.
I think the same thing is also relevant when people use chatbots to form opinions on unknown subjects, politics, or to seek personal life advice.
That's good