Part of the problem is that "Diversity" is being misused. It's not "diversity" if a company of 100 people has exactly the same demographics as the city it's in. That's homogeneity: diversity would mean that each company's demographics are wildly skewed in unpredictable directions.
Thomas Sowell has written about how intra-company diversity has harmful economic implications. For example, factories in New York used to segregate on religious lines--because if your Catholics are all off on Sunday, and your Jewish employees are all off on Saturday, then you can only work five days per week; all-Catholic or all-Jewish factories could do six days.