Thats what I am having trouble understanding. how does he know? Unless you take into account the context of them being Indian and his experience in India, they won't be able to tell whether its to figure out his caste, or just to have fun.
After you endure enough discrimination and persecution, your trust in others are eroded. You start to irrationally question the motives of others. Eventually you lash out preemptively because you’re afraid.
This is why discrimination and oppression is bad. It tears at the fabric of society.