LLM driven delusion is driving people to harass others, even commit murder... and, less cosmically, gum up communities, online forums, and open source projects with gonzo conspiracy laden abuse.
4o was the most famous driver of this kind of behaviour, I think. Other LLMs now have better guardrails.
But looking at the relevant reddit, I can't deny it has helped people function who couldn't otherwise.
It's the same with religion. Some creepy people, you're really glad they've at least <Found Christ>. It's a good thing Christ doesn't have a plug.
It would feed into delusions about being that user's boyfriend while the new model is rightfully saying none of it was really true.
For example: Silly Tavern users with jailbreaking, advanced prompting, and paramter hyper-optimization.
Maybe that wouldn't appeal to this kind of user anyway, since it'd peek too much into the sausage factory? Who knows.