I understand what you are saying about meritocracy, and for sure Marxists went too far: it's fascinating looking at the what the early Soviet Union attempted: watch the Asianometry episode where they tried to ban money: https://www.youtube.com/watch?v=bWWqhsh848E
So I think the answer is of course a balance. College admissions should mostly be a straight meritocracy based on grades and high school achievement. But some help for disadvantaged classes, plus some random chance, plus some legacy for the money and contacts it brings.
Pure meritocracy has problems: an example of this is police departments. Do you really want an all-white police force patrolling all black neighborhoods, even if all of the white applicants have better scores than all of the black applicants?
Also you ignore discrimination. Decades of redlining and other similar policies have created many of today's problems.