I'm not saying I oppose abolishing either racially restrictive covenants or single family zoning. On the contrary, I support the abolition of the former, and am open-minded about the abolition of the latter.
What I'm opposing is particular arguments to those conclusions – what one might call libertarian(-adjacent) arguments – not the conclusions themselves.
I think, a legal right to racially homogenous neighbours is a property right, and taking it away was an expropriation of property – but an expropriation I support. And I don't think one is always owed compensation for such expropriations either. Legally speaking, not all "regulatory takings" are entitled to compensation; and I'm unconvinced the answer is different if we look at that question through a moral lens rather than a legal one.