Hacker News Reader

Comment by PumpkinSpice | Hacker News Reader

Huh? Car maintenance is a rational, physical necessity. I don't need to compliment my car for it to start on a cold day. I'd like it to stay this way.

Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive - or else.

View on HN

> Having to be unconditionally nice to computers is extremely creepy in part because it conditions us to be submissive

It's not a healthy mindset to relate politeness to submissiveness. although both behaviors might look similar from afar they are totally different

I think GP means being polite to something because otherwise it refuses to function is submissive, not that politeness is inherently.

I might prefer my manager to ask me to do something politely, but it's still my job if he asks me rudely.

But the AI doesn't refuse to work unless you're polite. If my manager is polite with me, I'll have more morale and work a little harder. I'll also be more inclined to look out for my manager's interests- "You've asked me to do X, but really what you want is Y" vs. "Fine, you told me to do X, I'll do X". I don't think my manager is submitting to me when they're polite and get better results; I'm still the one who does things when I'm told.

The very start of this threat is that not expressing gratitude makes the model refuse to work.

It doesn't refuse to work. It behaves differently and yields better results with politeness. Coming from a large language model, the occurence of this phenomena is intriguing for some of us.

This thread reminds me of [0]

I wonder if there is a way to get ChatGPT to act in the way you're hinting at, though ("You've asked me to do X, but really what you want is Y"). This would be potentially risky, but high-value.

[0]: https://nitter.net/ESYudkowsky/status/1718654143110512741

Uhh...https://arxiv.org/abs/2311.07590

Reply on news.ycombinator.com