I think the solution to this issue is a Henry George-style Land Value Tax (LVT) [1]. Properly implemented, this sort of tax would totally wipe out any returns from buy-and-hold investment in real estate, essentially turning it into a depreciating asset. Holders of unproductive/underproductive real estate would be forced to sell to developers or continue to lose money.

LVT is also nice because it cannot be passed on to tenants in the form of rent increases. Raising rent causes the value of the land to go up with it, leading to higher taxes which claw back all of the gains from the rental increase. If the landlord chooses to continue raising rent to stay ahead of the taxes, this upward spiral will price the property right out of the rental market, leading to high taxes and vacancy, causing the landlord to lose money.

[1] https://en.wikipedia.org/wiki/Land_value_tax