I have a similar experience and I attribute it to the fact that when you are really an expert at something, others can pick up on it quick, maybe even before you yourself realize it.

For many years I thought I was a dummy for dropping out of college and just having a bad time with school and never getting good grades. Turns out I'm actually very good at math and I just had chronic depression from a bad upbringing. I taught myself calculus and then later, quantum mechanics, out of sheer interest and boredom after I was able to stabilize myself financially.