"A friend is training a simulated robot arm to reach towards a point above a table. It turns out the point was defined with respect to the table, and the table wasn’t anchored to anything. The policy learned to slam the table really hard, making the table fall over, which moved the target point too."

Seems as though the problem of learning unintended techniques sometimes may be better described as the model being too creative! Hitting the table is a really clever solution for the problem it was given. These examples show that the real challenge for researchers is constraining the models enormous capacity for creativity without stifling its ability to learn.