Humankind has transformed in the last century-and-a-half: a good chunk of our species is now taller than it's ever been.
Over the last 150 years the average human height has gone up in industrialized countries by up to 10 centimetres. And gains of up to 19 centimetres for Dutch men and woman.
Primarily due to improved nutrition, health, and in general a better quality of life. http://www.bbc.com/future/story/20150513-will-humans-keep-ge...
Pre-agriculture humans were about the same size (apparently, we can tell by measurements of the pelvic bone of ancient skeletons). We shrunk after we switched to agriculture, which basically started out as a steady supply of nutritionally inferior food (which could be indirect evidence that we mainly switched because we ran out of easy-to-forage healthier food). So a typical example of a 'good enough' solution, really.
I am a 6' Netherlander (among the shortest men in my family). I just traveled to Shanghai and met a lot of very short Chinese people - but I was surprised to see that there seemed to be a small population of some young men who were as tall or taller than me.
When I asked about these tall Chinese of a native, she explained that they were from northern China, and that they have a harsher climate and eat more fish and protein than locals. Many of the locals grew up subsistence farming under Mao, mostly eating rice, and she explained that the weak nutrition of the resulting diet resulted in less growth.
This is, of course, anecdotal. But in comparison to Dutch height gain, I haven't heard of a similar change in Dutch nutrition or health, at least not relative to peer countries.
Once everyone is everyone is fed the genetic differences become apparent.
There are plenty examples of this. For instance, the Normans who invaded England were nearly six feet tall. However, by the mid-17th century the average man in Europe was something like 5'3-5'6.
My great-grandfather was 5'3, and my great-grandmother was 4'11. They grew up during a famine in Hungary. My grandmother was 5'8, my dad was 5'11", and I'm 6'3".
Of course my mom is half dutch, and her mom was 100% dutch and was 6'1", which is huge considering she was born in 1913. So I definitely get some of my height from that side of my family.