This is a fallacy similar to hoping for interest rates to drop; these things don't happen in a vacuum, they are the result of other factors.
Rents might drop, but that's because the area is getting less appealing (much like interest rates might drop, because the economy is faltering). Is that net good? I don't know. Having more people living cheaply in an area with crumbling infrastructure doesn't sound great.
We only think that unusual because we've grown up in an era of perpetually-rising population. A lot of our economic rules of thumb don't hold once population growth reverses.
It is true that de-growth generates all sorts of other problems, though. One is labor: you may have houses, but they'll be shittily maintained, and you won't be able to find people to do the maintenance. Another is peace: when growth is happening, everyone has an incentive for peaceful trade so that they can capture the long-term profits of growth, but when growth reverses, the incentive reverses to conquering and pillaging as much of the pie now, before it shrinks, so you don't get left without a slice when it's gone.
Building more housing drops rent, but means the area is growing. People leaving because they are no jobs also drops rent, but good luck living in a place with no jobs.