Preagricultural peoples exerted reproductive control so they were not vulnerable kind of population crunches that affect other species.
Diverse food webs were also more reliable and nutritious than what was available to the early agriculturalists, as you say.
> "The bones of 'domiciled' Homo sapiens compared with those of hunter-gatherers are also distinctive: they are smaller; the bones and teeth often bear the signature of nutritional distress, in particular, an iron-deficiency anemia marked above all in women of reproductive age whose diets consist increasingly of grains"
from Against the Grain
see also: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3917328/
The linked article addresses famine more specifically.
I know a couple books[1] that suggest life prior to the agricultural revolution wasn't nearly as famine-stricken as we tend to think it was. These books claim hunter gatherers were quite proficient at finding enough sustenance.
Obviously starvation was still possible and likely, but was it the primary cause of death? Is there some more evidence that humans were suffering from starvation before agriculture? Wouldn't populations thin out and stabilize accordingly if starvation was such a problem?
[1] - _Sapiens: A Brief History of Humankind_ and _Sex at Dawn_
How happy are we who have never experienced that! How would our ancestors envy us!
Would our ancestors envy us or be happy for the fruit of their effort? There are setbacks and frustrations about the slow state of progress sometimes for certain. Maybe they would also be sad for the material orientation of many people and the loss of spirituality that gave them much hope and fortitude.
I for one think, this is a subjective question.
It certainly seems plausible to me, but I'd be fascinated to review the data and how it correlates to time and geography.