Focusing solely on consumers ignores negative externalities. There are many situations in which cost benefits from such externalities (slavery, pollution, human rights abuses, animal mistreatment) can lead to a favorable marketplace position vs competitors who do not exploit such externalities.
What is more, many of these exploits (slavery, pollution, human rights abuses, animal mistreatment) were or are legal at the time such exploitation occurred. Therefore using legality as sole criterion of propriety is insufficient. A moralistic/ethical take on business is, far from a negative thing, highly necessary.