Some form of "Group Selection" works, with stuff like the Green Beard Effect. There can be subgroups where sexual attraction is influenced by something like that. If group selection can work with a pair of traits in two halves of a population, it seems like it could work for a single trait in both halves of a population.
If you treat "race" as a word meaning "extended families that reproduce to some extent," then you have a great evolutionary argument for in-group altruism. By being slightly sensitive to race, you allocate slightly more resources to your own set of genes, meaning that people who share your genes, in turn, have more resources they can allocate back to you. In conditions of scarcity, that really helps. However, I don't know if it helps enough to deal with defectors (i.e. if there's a group that allocates lots of resources to group members, someone born with a "mooching" gene could afford lots of kids. It is basically the same math that wrecks cartels, just on a longer timeline).