I've been through the same. Grew up being told I could do anything I wanted, but people insisted that I focus on the stuff I was naturally talented at instead of learning the basic stuff I needed but couldn't figure out on my own. Like how to behave in a group.
The upside to it is that i have gotten through it. Somewhat. I've got a decent job, and I'm not worrying about losing it all the time anymore. I've got family, and who knows -- one day I might actually feel confident enough to make friends with someone.
But yes, it can't be said enough. Discipline, social skills, connections AND self-confidence are much more important than IQ.