Hacker News Reader

Can't an intelligent agent, artificial or otherwise, no matter how strict and out of band their orders, always be talked out of it?

View on HN

Cf. "Help, I'm a prisoner in a fortune cookie factory!".

Apparently something like this really does happen, although it continues to be hard to tell whether any particular instance is real:

https://www.businessinsider.com/chinese-prisoners-sos-messag...

I would go by how well humans do it, which would mean: "yes, you can probably talk it out of it, but when it matters, that's hard enough to do in practice such that the human/system can be used for important tasks"

With humans it's just that the stakes are high because you cannot generally have a human at your beck and call like we do with machines in general. If you limited to 5 questions per topic and and overall use time limit, you might see different input still.

Reply on news.ycombinator.com