Academia has become a prestige obsessed pissing contest. You need to go to a presitigous university for undergrad, so you can find a PhD position under a famous professor at another prestigious university who has lots of connections to the editorial teams of top tier journals, so you can publish at the prestigious journals, so that when you finish the PhD you can do 1-2 (or 5, or 10...) postdocs with other famous professors, so that by the time you compete for tenure track positions you use the connections with famous professors and publications at prestigious journals to become a professor yourself. Once you make it to professor, you need to use the connections and the prestige to get the big grants to hire lots of PhDs and postdocs yourself to continue the cycle.

If at any point you prioritise your drive and passion for a subject over the prestige and connections game, the peers who play the game will get far ahead of you in their careers, and you won't make it to professor. Saw too many 38+ year old postdocs who thought quietly pursuing their passion in their subject would get them anywhere.

No wonder all the people who just wanted to do their damn research went into industry. I did it myself and now get paid more than all of my former supervisor's postdocs combined. I don't think this even has to do with being a 'weird nerd'. Basically, at least in STEM, you have to specifically WANT to play the silly prestige pissing contest to stay in academia. Otherwise, forces of nature (economic and social), naturally push you out of academia.