Not quite, as long as we as a society want to protect the minorities. Either you force equal treatment, or things can escalate quite quickly:

- basic right of owner to rent property to anyone he likes without explaining the decision (and not to rent to anyone else...)

- basic right of hotel owner to accept anyone he choses (and deny access to anyone else)

- basic right of restarant manager to accept anyone he chooses (and deny service to anyone else)

- basic right of transportation owner to accept anyone he chooses (and deny service to anyone else)

- basic right of hospital operator to accept anyone he chooses (and deny service to anyone else)

Now pick your minority: it might have been Jews or the Slavic for the Nazis, it might have been the blacks in South Africa -- or the US, or homosexuals in Russia. Or the red-headed, the fat, the old, ...

Once you step onto this path, it's hard to put hard stop someplace