I vastly overestimated the impact of the former until university. Engineering classes kicked everyone's asses. Everyone was equally smart, but some people studied harder than others.
On the other hand, I know I am well above average at certain things because I've been doing them for many years. I have an above average interest in those things, and put an above average effort into learning them.
This is also true for other people. My dad is an average dad, until you ask him about cars or motorbikes. I knew he liked cars, but I didn't get to appreciate the breadth and depth of his knowledge until I started wrenching on vehicles. He spared me the details out of politeness, because I wasn't into cars. Now, he doesn't spare me, and it turns out he's a mechanical grandwizard. It was right under my nose this whole time.
I suspect most people are like that. They are average, except at one or two things. We tend to judge others based on the one or two things we value, and fail to appreciate them for what they're good at.