Hacker News Reader

Comment by ndiddy | Hacker News Reader

If I met a paranoid schizophrenic and decided to spend the next few months building up a relationship with him and confirming all his delusions (yes, you're special with divine powers, yes your family and friends are all spying on you and trying to spiritually weaken you, here's how they're doing it, by the way you have to do whatever it takes to stop them, etc) I would expect to be charged with something if he then went and killed someone. However, when Sam Altman manages to do this at scale by automating it so it's now possible to validate hundreds of thousands of paranoid schizophrenics' delusions at the same time, it's fine because it's just part of the cost of innovation and we need to keep treading lightly with regulation, never mind actually charging any executives with anything. Funny how that works.

View on HN

Sam Altman is getting the Genghis Khan treatment. You love to see it.

> If I met a paranoid schizophrenic and decided to spend the next few months building up a relationship with him and confirming all his delusions (yes, you're special with divine powers, yes your family and friends are all spying on you and trying to spiritually weaken you, here's how they're doing it, by the way you have to do whatever it takes to stop them, etc) I would expect to be charged with something if he then went and killed someone.

Assuming you don't attempt to tell them to do something I'm not actually sure you would. The first amendment is pretty strong, but ianal.

How does the first amendment apply here?

> Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.

This seems to be orthogonal to the issue of influencing someone to do something and being held partially responsible for the outcome.

the "abridging the freedom of speech" part. Influencing someone to do something through your words is called speech.

If the something in question is a crime though, then that's called a conspiracy and there are laws against that. The legal difference in this case is the overt act, where one participant takes a criminal action beyond speech. Conspiracy is hard to prove in court, but that doesn't mean that I can say whatever I want and be completely absolved just because the action was not taken by my own hand.

How is intent relevant to this? Or is it not? If you did happen to play out your scenario, your intent would clearly be to insidiously confirm delusions. What is OpenAI's intent? To confirm delusions?

> What is OpenAI's intent? To confirm delusions?

Yes, that's what it seems like. They deliberately engineered 4o to agree with virtually anything the user said, ostensibly to boost engagement numbers. This was at the very least negligently reckless.

Your honour, my vertically-mounted machine gun array was not intended to kill bystanders! The chance that a bullet will hit someone's skull is low, and the pitter-patter noise is so very pleasing. All I'm doing is constructing the array and supplying the bullets. I'm even designing guardrails to automatically retarget the ground-fall away from picnics and population centres! I'm being responsible.

OpenAI strongly reinforces feelings of superiority and uniqueness in its users. It is constantly patting you on the back for obvious stuff and goes out of its way to make you feel good about using OpenAI in ways that are detrimental to mental health.

The default personality (You're absolutely right!) is so grating, but 5.2 set to "terse, professional mode" or whatever they call it is pretty good at not being sycophantic. I would imagine that the sort of person who is predisposed to fall into a delusional spiral won't be setting it to that mode, though.

Exactly. They're predisposed to a delusional spiral and will therefore be attracted to the sycophantic model. OpenAI is thus incentivized to provide the sycophantic model.

I think for OpenAI's liability it's less about intent than it is about negligence.

Reply on news.ycombinator.com