Tim Ferris has some great things to say, but there's also the inherent risk that when working alone and taking a vacation, no one else is there to watch your back.
Culture at some companies is definitely more what like what you described, particularly at finance firms, but there are definitely cultural shifts happening right now as less college grads are going straight to finance every year, allured by the perks that tech firms have to offer such as unlimited PTO in some cases.
It seems like your situation is very different from that though. Working at a small startup is really hard, particularly when you already have a family. I wish you and your startup the best of luck in the coming year.