The discussion reminds me of Karl Sims’ Evolved Virtual Creatures. In the Siggraph talk he mentioned that they would absolutely exploit any bugs in the physics simulation that they could. That’s all that’s happening with this article’s LLMs - the metric is dictator, and simply and predictably leads to any and all possible behaviors that achieve higher scores.

https://www.youtube.com/watch?v=RZtZia4ZkX8

You know what’s really weird though? In a very long and detailed and thoughtful post about how AIs are behaving in surprising ways and doing something other than what they “say”, it is non-stop anthropomorphizing the AI in practically every sentence. The number of silly statements about what Claude “thinks”, or that it “fights”, what it’s “choosing”, how it’s “pretending”… it goes on and on and on speaking as if it’s human. It’s not human. The very problem here is projecting meaning onto a machine. LLMs are statistical token generators and they don’t have values or agenda, and don’t know what faking is. The analogies are specious and hyperbolic; they’re tempting to believe. Sure some humans do fake things sometimes, but not for the same reasons. Everything having to do with thoughts, opinions, values, and alignment, come from the training data, and specifically our interpretation of the training data, which the LLM can only fake because that’s how it was designed.

The behavior isn’t the least bit surprising when you think of it as a machine, it’s only surprising when we treat LLMs as if they’re people. And the more we buy into the anthropomorphizing, the worse this problem gets.