Your competitor might find it advantageous to compete with you on employee wages and poach your good employees leaving you unable to compete. Supply and demand works for salaries as much as it does for physical goods.
And what about the poor bastards who are judged to be "average" employees by whatever metric?
Which is what history proves happens all of the time, and is the reason that we don't have any labour laws, right?