It's hard to weigh the aggregate harm of doing something slightly bad at massive scale against the aggregate harm of doing something very bad to a small group. Both are pretty bad, and we should be vocal about both.
Ethical AI are rules of engagement for honest brokers. Having honestly intentioned people not have hidden badness in their products is still important even if other dishonest people do bad things.