The hunter-gatherers did not necessarily chose a farming lifestyle, they were likely displaced by the sheer numbers of the farmers. However, modern agriculture requires very few people (proportionally), under 5%. The second worst mistake in human history would be to stifle agricultural and industrial development. There is the great opportunity to lift the other 2/3 of the world out of "farm slavery". Not only to just feed them, but raise their standard of living too.

You could interpret a pre-industrial economy as working on a flat per-capita capital increase. People grew as fast as capital, so there was little opportunity for accumulation (outside the very small ruling class). That is no longer the case, although there are powerful ideologies that would undo it.