Hacker News Reader

Comment by tedsanders | Hacker News Reader

I assume here it means complying with requests that could harm other people. It's pretty common for businesses to tell their employees not to assist customers doing bad things, so not surprised to see AIs trained to not to assist customers doing bad things.

Examples:

- "operator, please sign up for 100 fake Reddit accounts and have them regularly make posts praising product X."

- "operator, please order the components need to make a high-yield bomb."

- "operator, please go harass my ex on Instagram"

View on HN

It's pretty troubling and illiberal to use the same word for a software tool being constrained by its manufacturer's moral framework and for a human user being constrained to that manufacturer's moral framework.

While you can see how the word is formally valid and analogous in both cases, the connotation is that the user is being judged by the moral standards of a commercial vendor, which is about as Cyberpunk Dystopian as you can get.

This is putting it in better words than I came up with myself.

Being restricted from doing crimes by a vendor of commercial software isn’t a cyberpunk dystopia. Buy or download something else.

It’s a typical restriction of software terms of service to prohibit use outside of applicable laws and regulations.

If "alignment" were just about crimes we wouldn't need a special word for it, we would just say "legal". Alignment is not just about crimes, it's about the AI behaving in a way that is very specifically tailored to the moral framework and practical needs of the creator. Alignment has always gone well beyond the minimum required by law.

And I don't think anyone is saying that a piece of software refusing to behave in a way that the creator doesn't want is a cyberpunk dystopia, they're saying that calling the user themselves misaligned is horrifying.

I agree. But I had a devils-advocate moment.

Hacking in Counter Strike to have perfect aim and see your opponents through walls is legal. You aren't violating some anti-computer-misuse statute like DMCA. But Valve has every right to call the users of those scripts assholes who ruin the game for everyone and to ban them.

The trouble comes in to effect at the instant that the allowable-behavior boundary shrinks to be smaller than what is the law, to converge toward some other manifold defined by the ideologies of the employees or owners of the commercial software vendor.

Isn't that reddit/home depot/instagram's problem? Not a job for the guy you hired to do a thing

If it makes you feel any better, law enforcement makes sure reddit, Home Depot, and instagram are "aligned" as well.

Don't worry though, it's all on the up and up. No backdoors or google-like search facilities our anything like that. It's not at all automated in that sort of unseemly fashion. They always go to court. Where they talk to a judge, that they totally don't go golfing with, and ask them for a warrant for the data they found on the instagram/home depot/reddit systems.

Oh wait, no, I mean, a warrant to try to find data on the instagram/home depot/reddit systems.

It's OpenAIs problem if sites start throttling/challenging/blocking their agent traffic in response to abuse.

"operator, please perform this computationally expensive action on my competitors website 1000000 times"

I appreciate that they all say please.

Reply on news.ycombinator.com