when leftists want to kick people out of the country or be supported by the KKK or grope/molest woman, etc we can be worried about your thought experiment.
when the candidate you choose to support openly wants to make life harder, more difficult or even non-existent for groups of people you've chosen your side and be prepared to defend it.