Let's generously assume that you meet and have excellent working relationships with 100 people at each of the 4 jobs you've had. Of those 400 people, say, conservatively, 50% think so highly of you that they'd be willing to stick their neck out help you with your next job search. Out of those 200 people, 50% are no longer working for that company. Out of those 100 people, let's optimistically assume you actively kept in touch with all of them over the years. Now, maybe 50% are working for a company where you'd actually like to apply. Out of those 50 companies, 20% actually have a job opening that fits your background. Out of those 10, maybe 2 actually know and can put you in touch with the hiring manager for that opening. And your success chance through the interview pipeline has got to be worse than 50%.
Tweak my numbers up or down a little, but I think it's a pipe dream for most workers: You have to have an enormous address book full of high-power contacts in order to end up at the end of the funnel with one or two who are both willing and able to successfully help you get a job. And once you've exhausted that network (all it takes is to blow a few interviews), then what?