"You are much more smart because your skin is lighter than I expected," now that would be racism.
Or "I will be happier to work with you because you are lighter than expected."
Or "You must have gotten that job because you are lighter than expected."
If you’re making the comment, it’s because you’ve internalized a few stereotypes around skin color, and made an assumption based solely on those stereotypes. Stop yourself at the point where you’re making the assumption.
Probably each person has some mind-image that’s more common than others (probably guided by early childhood experiences, or mirror image of you), or even if you’re on average unbiased there’s still going to be some image you have in mind for each particular person, which is a perfectly legitimate reason to be surprised upon finally meeting that person.
Honestly the only reason this is controversial is because of “skin color” and the current political climate.
“Oh, I’m surprised, I imagine you having red hair.”
“How come you imagined me having red hair?”
“Don’t know, just because.”
Inherent in assuming a skin color is the implied association between, for example, intelligence and skin color.
"I'm surprised you're black," is not so far off from the acknowledged insult, "You're really $X for a woman."
WRT your example, anger issues are stereotypically associated with red hair, so there could be an implied "you have anger issues" that goes with that. "Oh, no reason," is also a very common deflection when being called out for poor assumptions or implied insults.
There is always a reason associations between mental traits and physical features exist, even if you're not consciously aware of it.
EDIT: And to go a different direction, if someone truely doesn't make such associations... Now that they know some people do recognize those associations, just be polite and don't make such comments about people's appearance - be it skin color, hair color, height, weight, etc.
When you meet a software developer, saying they're unexpectedly dark strongly implies you didn't think someone of that ethnicity was capable of being a software developer.
It doesn't need to be spelled out -- the implication is very clear, and if an employee claims to be ignorant of this, they should at least be sent on the race awareness training.
"Assume the worst" = This is precisely the kind of mind reading that leads to a paranoid, unhappy, eternally vigilant society.
I don't see the difference between that and realising someone is a different ethnicity than you expected.
Racism is (by definition) prejudice, discrimination, or antagonism. Being surprised because someone looks different than you imagined is nothing like that.
Because they're of an unusual height, I assume. You had an expectation that they were of average height.
You can look at the pictures of the person that received that comment. They don't have an unusual skin color, and the ethnicity matches their name (say, they don't have a Russian sounding name with really dark, almost black skin). It shouldn't be unexpected to see their skin color.
Another option is that the expectation didn't come from averages nor names, and it came from prejudice about how software engineers look like. So yes, it definitely can be interpreted as a racist remark.
My color of skin definitely depends on the camera I sit in front of. The best cameras are as good as a mirror, but lower quality cameras built in laptops are awful. Looking at greenish or greyish myself is a bit nauseating.
Everything is blue if you wear blue tinted glasses
I live in a city where I expect everyone to be expectedly dark, and anyone that come near to my white shade is very unexpectedly light
If I go to the pale end of the country, any darker skin would be unexpected, no matter the job title and no matter the IQ level
No unexpecting person should be liable for your projected meaning on surprising turn of events