Apparently we're close enough to post-scarcity that people forget jobs are supposed to do things and not status symbols.

The point of meritocracy isn't that it's more fair or ethical, the point is that we give the most important (on the margin) jobs to the people most capable of doing said job. The fact that money, status, and power flow to the people doing important jobs is an incidental side effect.

Now universities aren't exactly jobs, but the point stands. Universities are supposed to educate people, so that those people can do stuff. Letting in people with connections and not ability means we are not making the best use of our limited educational resources. Note that this is an argument for affirmative action as advertised and an argument against affirmative action as practised.