I am confused by how this is supposed to be useful. It seems like the researchers are defining side-effects as things that "disrupt the world" (of this life game) and training an AI to avoid this.

But this seems like at best one of a whole host unexpected effects one might consider. AI that discriminates in a way that society frowns on might not "disrupt the world" in such a visible fashion.

I don't see how one can get away with an entity doing stuff for you with that entity understanding your model of the world.