My social circles don't have much overlap with, for example, startup founders. Developing personal relationships in my network is not likely to ever benefit my career growth. When people give the advice to "network" to enable my career growth, the people who would be most helpful tend not to have much in common with me, and building relationships with them often means not being true to myself, or even hiding myself.
It isn't enough to simply build relationships or even be good at building relationships. Mostly, one needs to buy goodwill from those people who actually have power to help you.