It's not about references. It's about building a network of colleagues who respect you and your work. Many years ago, when I started doing consulting/contracting work, literally all my of my jobs came through people I had previously worked for or with across a variety of companies. And if you play your cards right, as the years roll on, you won't even have to apply for jobs other than as a formality. Instead, people who's respect you've gained will try to bring you into where they work.