I spent several years holding down a regular job and doing the freelance work at night. Eventually, as I successfully completed those jobs/tasks, the word spread and I had enough coming in from the night work that I was able to shed the full time job and go on my own. Yes, it means spending a few years of working two jobs, which isn't a luxury everyone has or even wants to exercise. It's definitely paid off for me though, as I've now been 100% freelance for about 5 years.
My other suggestion is to see if you can convert a portion your existing full time job to freelance (i.e., convert from an employee to a contractor for your existing job). I did that as well, which helped fill in my client gap for a bit.