I believe the principle is that it is inherently bad that _anyone_ owns a second home.
Pay no attention to all those failed states that have already tried this - this time will be different!
Further, your argument doesn’t hold water - food is a requirement, but does it have to be grass fed beef? Could I not make do with soylent green?
This way of thinking is a race to the bottom. It must be avoided at all costs.
What if I own a family farm and want to build a new modern house on some of the property? Do I need to demolish the old one, sell it, be homeless to build a new one?