A nightmare scenario for LLMs is becoming another dealer of cheap dopamine hits, using your personal history, your anxieties, and whatever else it can infer from you to keep you hooked.
Not just a danger for interpersonal relationships, it will enable everyone in a management structure to surround themselves with perfect yes-men.
I asked ChatGPT to isolate individual chats so as to not bleed bias across all chats which funnily enough it admitted doing so.
When I asked Grok, it said it is set as the default out of the box.
If it had access to its own settings, and it wasn’t making things up, and it wasn’t lying… but why would it be trained on any of these things?