As a mostly introvert I can say that I lose energy from interacting with a group, and gain it back when I can be alone. But at the same time being social relieves stress. So it's a push/pull.
I actually really like most people. But I like them one on one when I can have a real conversation. I hate party chit chat because I'm terrible at it.
I have a couple friends whom I'd call extreme extroverts. One big difference between them and me is if they try to initiate talking to a stranger (at say a bar or some other reasonable location) and get rebuffed or otherwise have a bad experience, they instantly forgot about it and move on - whereas I'm still thinking about it years later.
So yeah maybe introversion is just social anxiety.