Being for or against affirmative action and quotas is kind of orthogonal to being a racist, though? There are certainly racists who support affirmative action, and obviously ones who don't. (for instance, people who think that there is a difference in mean IQ between races, but thinks that affirmative action to compensate would increase societal stability) Similarly, there are people who believe all racial differences are skin deep / stop at the neck who support affirmative action and ones who do not. (For instance, people who think that affirmative action is counterproductive and will lead to racism)

There might be a correlation there, but let's not pretend that only two corners of that matrix are populated.

I'm not an advocacy for affirmative action and quotas, but I think they were made to "counteract" social discrimination based on race and sex, and not race and sex differences themselves. I think I might say that in an imaginary society without racism and sexism, I would be against affirmative action and quotas! (well there wouldn't be need for them...)