I've been independent going on 8 years now, I started with letting some trusted contacts I had made in the industry know, and one of them came to me with an opportunity a few months later. It was a terrifying jump at the time, but I have no regrets and was glad I pushed through my fear.
These days I still do direct independent client work, and new clients come through a consulting / MSP firms I work with, and from my existing network. I get better paid with direct work, but the work coming from the firms is usually more interesting.
Eventually you get to the point of turning down work just because you have too much already, and deciding what you really want to be doing with your time.
I'd say the most difficult thing is trying to find that sweet spot between too little work, and too much work. Sometimes the sales pipeline from initial interest to actual engagement can be measured in years.