You're making the presumption that people adopted agriculture because it "gave them more", when there could be a vast number of other reasons for adopting it.
More than likely, the first tribes that started becoming farmers did so because the area they lived in become bad for obtaining food (whether due to the extinction of big game events, or moving to a naturally inhospitable area). Because of the nature of agriculture, this gave them a lot more power, and they started conquering tribes.
When your choice is between "get slaughtered by the farmers", "have your food supply diminished by the farmers", or "become a farmer", guess which choice you pick?
This is all speculation, though. I'm merely trying to get you to realize that there are many reasons for technological progress (esp. the adoption of agriculture), the least of all being that "it's human nature to want more".