This is backwards. If society gets more expensive because we no longer oppress certain groups in the same ways, then it should have been the case that people from these groups were the ones getting cheap housing in the past since they were “buying” a shitty social experience.
But that’s not the case. It was straight white families who got cheap housing while at the same time holding minorities back.