You're simplifying to the point of nonsense. Freedom you say? How about the freedom to have a say in government of the place you're living in? That seems a pretty fundamental freedom. When the rich folks of a town vote for planning restrictions and the vote goes through, that's an expression of freedom.

Sure, we both don't like it. We both agree it has bad consequences. But what I'm trying to say is that there's a real want backed by serious money. One way or another, it will create a market (maybe a shadow market). Rich folks will always want "no poors in the neighborhood" and will keep trying to find ways to spend money to ensure it. They'll never give up.

That's why I'm trying to think of solutions that don't require arm-wrestling one market vs another. For example, if we somehow created jobs elsewhere so that poor people wouldn't have to fight rich people for city air, then maybe that could work too.