Yes, as the author stated, what is considered "average" depends of the group, so I guess he won't be hiking with beginners or overweight people...
Your analogy is off : there is no challenge in cooking faster than others and no adrenaline rush either. What if you are a vegan & asked to cook burgers with a group of hunters? That is what the author tries to say : go hike with people of your hiking level, to avoid frustration on both sides.