This line of reasoning is very interesting to ponder though. The black plague in Europe made survivors materially richer as they were able to take possession of what was left behind from the deceased. As there's less and less people to compete for stuff, the existing younger populations will be richer, thus giving them an incentive to focus on leisure and reproduction. Then there would be a cycle were the sharp decline in population, plants the seeds for the next big rise.
It doesn't address what that time frame would be or what happens in the interim. When you say pretty quick is that 5 years or 500 years?
While I agree there are some strong feedback mechanisms that lead to population growth, stating that alone ignores the fact that there are even stronger forcing mechanisms in the present pushing population down. The outcome will depend on how those extremely strong feedback mechanisms on both sides interact moving forward.
The modern world has yet to see a population where growth rates have bounced back, and some countries like Japan have been declining for more than 50 years.