A lot of this hinges on whether you believe you should be exempt from your own moral standard.
The idea of "racism" as requiring a position of power comes most directly as a reaction to the "reverse racism" bullshit from racist apologists who want to roll back affirmative action and various other programs. The reaction comes from sympathetic academics and writers who would rather focus on words and philosophy, and are better at that, than practical issues "on the ground" so to speak. And yet, AA is an example of something which is known to be imperfect, but is needed in some practical sense -- much like the tracking of race or gender ratios in various careers. I think it's a huge mistake to attempt to redefine words in terms of power, because it plays into the corrupting idea of power as the ultimate goal of human endeavor.
The reverse racism shit is really dumb, but the proper response is that there is no such thing as reverse racism, just racism, and that racism is borne from improper over-generalization and a unwillingness to examine ones own prejudices about the world. It is not borne from attempting to correct for those things in other people (e.g. AA, etc.).
Racist views may be understandable as a reaction to oppression by the oppressed, and you make allowances for human frailty -- it is still not generally acceptable. Redefining the word "racist", or any negative epithet, such that it can never be used against your particular group, whoever they are, is just an invitation to corruption.