Yet if a business used that line to refuse service to a minority, a religious person or atheist, a pro-choice advocate, or a LGBT person (in a nation where these are not protected groups or in a time before those groups were protected), you'd be up in arms about it even though it's the exact same excuse. See the double standard now?
The groups you oppose would absolutely love to be able to use that line of reasoning to defend themselves (remember the Colorado baker case?). The irony of people calling themselves liberals trying to use it to put an aura of legitimacy over their behavior too is delicious.