Hacker News Reader

Comment by Extropy_ | Hacker News Reader

How is intent relevant to this? Or is it not? If you did happen to play out your scenario, your intent would clearly be to insidiously confirm delusions. What is OpenAI's intent? To confirm delusions?

View on HN

Your honour, my vertically-mounted machine gun array was not intended to kill bystanders! The chance that a bullet will hit someone's skull is low, and the pitter-patter noise is so very pleasing. All I'm doing is constructing the array and supplying the bullets. I'm even designing guardrails to automatically retarget the ground-fall away from picnics and population centres! I'm being responsible.

OpenAI strongly reinforces feelings of superiority and uniqueness in its users. It is constantly patting you on the back for obvious stuff and goes out of its way to make you feel good about using OpenAI in ways that are detrimental to mental health.

The default personality (You're absolutely right!) is so grating, but 5.2 set to "terse, professional mode" or whatever they call it is pretty good at not being sycophantic. I would imagine that the sort of person who is predisposed to fall into a delusional spiral won't be setting it to that mode, though.

Exactly. They're predisposed to a delusional spiral and will therefore be attracted to the sycophantic model. OpenAI is thus incentivized to provide the sycophantic model.

I think for OpenAI's liability it's less about intent than it is about negligence.

> What is OpenAI's intent? To confirm delusions?

Yes, that's what it seems like. They deliberately engineered 4o to agree with virtually anything the user said, ostensibly to boost engagement numbers. This was at the very least negligently reckless.

Reply on news.ycombinator.com