I think a lot of it comes from the modern definition of racism involving power + prejudice. As a group Chinese people appear to be relatively successful in society (certainly in tech) and so sit slightly higher up the power scale than black people do. So if a white person makes a negative comment it's going to appear as societally "punching down" much further if it targets a black person vs a Chinese person.

Basically it seems worse if you reinforce stereotypes against a group that is already struggling.