Admissions offices curate the incoming class. In doing so, they have multiple objectives: maintain a high standard of intellectual ability and engagement, admit a heterogeneous student body so that people are exposed to others who are not like them, and preserve the particular character of the institution (a specific brandy of nerdy at Columbia/Chicago, the hacker ethos at MIT, etc).
An admitted class that's mostly homogeneous (in race, interests, gender, whatever) is just as much of a failure as an admitted class that's stupid.
The purpose of an elite college's admissions office is not to admit the top N students on some metrics, nor is elite college your "just reward" for being in the top N percent of your year on some metrics.
As with tech employment, there is an element of having the specialization an employer wants, an element of "culture fit," and an element of being able to do the song-and-dance that appeases the gatekeepers (the SAT is a lot like a coding interview).
As with dating, it's not "academic performance in, admissions out." The college has a personality too, and it has to like you.