If, for example, you made admission to elite colleges(which is a whole other can of worms) ignore ethnic backgrounds, you'd get a result where far fewer applicants of certain ethnicities would be accepted. In an arbitrary sense, this could represent "equal opportunity" if you believed that these people materialized from the ether the day they submit their application. But if you take into account their lives leading up to submitting their application, "equal opportunity" requires counteracting powerful structural factors that make it harder for people of some backgrounds to get into college.
The reason Harvard admission of Asian students is cited to such a tiresome degree is that it's a rare case where there isn't a clear upstream justification. That's why there's a big lawsuit and Harvard is getting tons of awful press about it. But it also lines up with the fundamental problem we're trying to fix with affirmative action. Harvard and its admissions systems were invented by white people, and they have proven to relentlessly favor white people, who then have easier lives and more money they can use to pay SAT tutors to help the next generation of white kids get into Harvard. It's not surprising that Harvard would warp the affirmative action system away from its original purpose, to correct for historical oppression, in order to... help white people get into Harvard. The fact that Harvard is discriminating against Asians is not a reason to implement a fake-fair system that has the end result of discriminating against all other non-white people.
It's theoretically possible that there are some other cases somewhere that show large inequality of outcome but have no unfairness upstream. But I'd rather have the task of discovering and fixing those rare cases than the present system, in which being born with a certain background gives you radical advantages or disadvantages on the basis of, basically, ancestral violence.