If I couldn't immediately answer the question, I felt was just stupid and it wasn't worth my time.
This, in spite of a family insisting I was the greatest.
Some would argue it's because of your family insisting you were the greatest, in part.
>At some point I decided if I had to work at anything, I didn't want to do it. If I couldn't immediately answer the question, I felt [it] was just stupid and it wasn't worth my time.
Yep, you are a smart, great person. Your family loves you because of all of the smart, great things you do, in part. If something is hard and doesn't come naturally to you? It must be something stupid and not great. So don't worry, you are still smart and great, and people still love you. Don't try anything hard.
I'm obviously being facetious, but according to some experts (though not all, not even most), that emotional dynamic is encouraged by praising kids for their intelligence or abilities, instead of effort. I don't know how much I believe it, but that type of weird insecurity does seem fairly common among intelligent people who were extremely precocious.