Anecdotally, I feel like the second-time founders see VC as a very specific tool and many prefer not to raise... They'd rather bootstrap and scale more slowly, retaining control, freedom, and flexibility. Granted, a lot of founders also have bad experiences with VCs.
I think your framing VC as a tool is the right one. Not sure if first-time founders are still like this, but ~15 years ago, it seemed like raising VC funding was validation in and of itself and you were perceived as successful due to the raise. The second-time folks know this is neither validation of your product nor is it any sort of guarantee of success... If you need to raise lots of money to capture a speculative market or build something that won't see revenue for a while, VC is great. If you want to build a decent business that grows 20% to 30% per year and compounds over decades, then maybe it's not for you.
Finally, VC is a broad category. Raising money from former founders or from strategic VCs who can legitimately mentor you is very different than getting a check from a fund that makes 100s of investments per year and simply manages you as an arms length portfolio entry.