Hacker News Reader

Is it stupid and useless at following simple instructions without going off the reservation and doing stuff you don't want it to do?

No, it's "cheating" which makes it scary and smart even when it's not doing something useful that you actually want it to.

Like when Teslas try to swerve off the road. It's not failing at driving in a straight line, it's just trying to cheat by taking a shortcut through the bushes. That's how smart the Autopilot AI is.

View on HN

Except these AIs are both scary and smart. I think the characterization is warranted.

The whole point of calling it “cheating” is to warn us that if we tell an AI to do something, we need to take into account that it may well do something we don't expect; something that we as humans dismiss as “cheating”.

Someone in this thread gave a good example: if you ask an AI to get a reservation at a restaurant, and make sure the reservation is in place before exiting the agent loop, you don't expect the AI to hack the restaurant and erase someone else's reservation to make space for yours. But an AI will absolutely do that.

Reply on news.ycombinator.com