https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ is the essay that crystalizes the reasons that Selection and Markets are not the forces we want to be aligning AI (or much else).

In short (it is a very long article) fitness is not the same as goodness (by human standards) and so selection pressure will squeeze out goodness in favor of fitness, across all environments and niches, in the long run.