Usually the last name is a giveaway. That said, last names are very regional in India, so someone from north India can pretty much instantly infer the caste of another north Indian, but that person would have difficulty doing the same with a south Indian last name.