The same goes for teachers in general - you can have professors who are very smart, but are poor teachers. It's a completely different skill set... that's why it's so exciting when you find someone who has both!
What usually convinces me that we're reaching into the "I'm just not smart enough" territory is when I listen to multiple explanations from multiple sources and I'm still stuck (although ... I find that on certain subject people are just parroting each other a lot, which makes me often question whether the person doing the explaining actually understands the topic at all).
As an extreme of this, if it takes you 20 years of hard study to grok a topic that normal people absorb in 6 months ... harsh to say, but you don't fall in the "smart" bucket.