People weren't their strongest even when most did manual labor. Now that humans are free from mental labor we work on creating and optimizing the best exercises for each mind. Couple that with restructuring transport infrastructure and diets many people will be smarter and fitter than at any time in history. They won't be able to outrun an automobile or out think an autointelligence.