The purpose of affirmative action policies is to help under-represented groups. Asians aren't under-represented at Yale, they are 19 percent of the student population, while only 6 percent of the American population.
I will answer your question with a question:
If the Supreme Court requires universities select their student based solely on standardized tests, and consequently this results in our top universities being made up of primarily of foreign nationals (there are billions of Chinese and Indians, and it is illegal to discriminate based on national origin), would you favor that inevitable outcome?