Humans kick people out far before the stakes are that high. I'm reminded of how people with Leprosy were treated before there were drugs to treat it.
Generally speaking "hanging around with the least fit members of your species" is a losing strategy if you want to not starve long enough to reproduce so it should come as no surprise that evolution selects for the trait of avoiding the least fit members of one's species to varying degrees (depending on the specifics of the species and the type of "least fitness" in question).