I was in your position 2 years ago but without a family to support. With a family I would be cautious, have a run away of 12 months salary or have some working contracts ready to start before leaving the work.

Also, expecting a constant flow of 100-150$ freelance work can be difficult, and although you might think that you just need to work 2 days a week, when you have a company you have lots of other stuff to deal that take lots of time like meetings, emails, social security, taxes, accountability, project budgets etc.