Plenty of white people are uncomfortable around minorities. Do they all love that feeling? I'm confident most feel intense guilt about it and never act on those impulses in any way. Just having a 'racist' feeling doesn't make someone racist.
I would argue that that is exactly what racism is. It's having an irrational feeling towards someone and assuming they have characteristics based solely on their race. Why is it so hard to judge people on a case-by-case basis? By your own admission you say that these personal anxieties are irrational and then you go on to attribute it to being a minority. If instead you attribute it to yourself personally, it becomes a problem that you can solve for yourself rather than an affliction of which you are victim to. There are plenty of minorities (myself included) who don't feel this way.
No group dictates the characteristics of the individuals in that group. Instead, the emergent quality of individuals that belong to a group can give that group a stereotype. However, being the unique individual that you know you are, you're capable of shattering any and all of those stereotypes! You're right that you can solve it for yourself; focus on that opportunity/growth mindset! Don't let anyone tell you how you should think/act because of your group "membership"
But it very well might be (biased against him).
Racism exists. Racism is widespread. People in populations that are targeted by racism are much more familiar with what racism is present than those of us in untargeted populations. Much racism is subtle and implicit.
If you can logically accept those statements, then I don't see how you can claim to know how likely the presence of bias might be. And worrying about something that could actually be true is not racism. Even if the reasons involve race. Assuming that the interviewers are thinking X or will do Y because they are white is racist. Worrying that they might is probably just being realistic.
Again, though, I am the one making this argument that you are declaring to be racist. The person you replied to wasn't. He was just expressing an anxiety he experiences, and wasn't even speculating on how likely that anxiety might be based on reality.
Pretty glaring assumption there.