Broadly true, but shorter periods of food deficit leading to decline in population are still possible.

The last agricultural famine in England was only in the 17th century.

> Through virtually all of history, there's a strong correlation between "more people" and "better living conditions".

That's survivorship bias. Populations always try to grow, not to shrink. However, growth will only occur when possible (due new technologies or more resources being available).

You’ve just never lived through a famine.