That is because the other ethnic groups are not the dominant ones in society, and are typically disadvantaged on the whole because of their race. My personal view as a White person is that I have enough advantages by my White male-ness that I do not need to demand more, or do anything to offset whatever minor inconveniences might arise from the same. Doing so would seem petty to me if I walked in another man's shoes.
Is this racist? Yes, I suppose. White privilege is real, though. A recognition of this fact requires some ability to discriminate between race when observing the state of society, so perhaps "racist" should not be the bad word. Instead, "racial oppressivism" might be a better thing to demonize.