This is _only_ because, in medieval times, when there was enough food people had more children until there was no longer enough food for everyone, maintaining the equilibrium and making it to be "just enough" most of the times, with a lot of "not enough" between those. Incidentally that is also how most animals and normal evolution operate.
But what if they only have ~2 children per couple, so they CAN easily maintain this lifestyle? Specially given they are long-lived (taking care of things long-term) and have culture+technology either from abroad or from long ago (mills, craftsmen, etc). I can definitely see this more plausible than the jump to classes the post does.
As a personal note, once I visited the UK countryside and was surprised by the amount of wild berries there were, we grabbed a bunch and there was definitely enough that we could've eaten for few days. Sure probably not a whole town and not all year round, but these were _just_ the wild ones, with a bit of work I can see how lush the land could become.