The issue is that Americans with half-baked knowledge think that all Indians have a caste assigned to them, or that India has a national level caste allocation policy. For Americans who hate India, this gives them more ammo to hate us.