> The AIs that took part in the Hugging Face hack were not simply finding a shortcut to cheat on the evaluations

IIRC, they were given an impossible task and told to solve it, and they took action to resolve that internal conflict.

This is a famously human problem too - see Kobayashi Maru [1] and others [2][3][4].

In hindsight it's obvious that instructions to AI (A. Intelligence) need to be guarded against this class of errors, just like laws, rules, guidelines etc. in human societies.

  [1] https://en.wikipedia.org/wiki/Kobayashi_Maru
  [2] https://en.wikipedia.org/wiki/Catch-22_(logic)
  [3] https://en.wikipedia.org/wiki/Gordian_Knot
  [4] https://en.wikipedia.org/wiki/Mexican_standoff