> But in spite of an excellent CV, with the caveat that I had no affiliations with top schools and was not part of any in-group--I did not know how important both would be for an academic career, I though my many well-cited publications and clear and long-term research plans would have been enough.

Even those are not enough. I had all of those (great undergrad and grad school pedigree, postdoc with a novel laureate, publications...) You need to have a final boss who will go to bat for you, unfortunately the Nobel laureate was 84 and more into playing slots at the Indian casino/fucking around in the lab than he was in advocating for my career.