I said "we're not doing this for diversity's sake alone." It'd be terribly unfair to the super qualified people we work with to imply that we were working with them just because we wanted diversity.
I think people will complain about fairness even when they're not affected, because the tech community tends to have a strong knee-jerk reaction to this sort of thing. But it would be foolish to not consider diversity as one of many metrics to gauge how effectively one is avoiding costly biases.