Another factor is that crops have changed vastly since the start of agriculture, which is also a major factor in the relative growth rate of human population (along with disease, which arose hand in hand with agriculture, of course) - see Teosinte vs. Corn, and Emmer Wheat vs. Durum. Same plants, shaped by man's hand since time immemorial by selective breeding.
So, yeah. These guys actually had it really well off compared to historic humans, as they had agriculture, but it doesn't function well in isolation, particularly in such a harsh environment as the taiga - don't forget the place used to be inundated with hunter-gatherers before everyone migrated for the cushy disease and war-ridden life agriculture offers.
Survival was a bitch until we figured out farming and trade - and it's improved over the last 150 years or so due to modern medicine (lower infant and adult mortality), improved crops (green revolution), improved productivity (industrial revolution), and all the rest.