Actually caste is far more tangible than something like skin color. It is encoded into birth certificates etc in India and is immutable. You can change religions but your caste carries into the new one.
Ref https://www.nriway.com/nri-service/nri-birth-certificate https://www.indiafilings.com/learn/caste-certificate-in-indi...