Similarly, we have little in common with immigrants from poor countries fleeing extreme poverty and dysfunctional violence. In the immigration analogy, we are more like the plutocrats buying access through investor visas. Even the most pro-immigration liberals don't think immigration should be for sale to the rich.
So why should cities be forced to remake themselves for our benefit?
The simple answer is, it's not for our benefit. The highly skilled, well educated, and well employed will happily take over every existing housing unit, no matter how poor its previous occupant. We might even prefer to live in beautiful, low-rise cities with only fellow elites for neighbors.
Increasing scale until there's enough for everyone is the alternative to just replacing existing residents in existing homes.
Those type of buildings are not very profitable to build now with modern building requirements like ADA. Once the economy goes south, the rent and quality of tenant will plunge.
Real estate is always boom-bust. Having a conservative outlook about building is pragmatic.