As someone who started a company in my 30's and is considering another one in my 40's, my advice to you is that you just have to be more pragmatic. I have a wife, kids, mortgage, etc and I spend more time vetting the idea than I did in years past. I talk to more people about the idea to see if I'm onto something or if this is just an idea destined for failure. By spending a lot more time ensuring that the idea has legs I'm giving myself more data and also laying the groundwork if it is something worth pursuing. I would never quit my job unless I knew the startup could sustain my livelihood for at least one year.

So the short answer is that you can absolutely do this. It's about how you approach it and ensuring the people close to you are on board for the ride as well.