My husband and I run a SaaS/infoproduct business which grossed right around $600k last year and we do not work long hours (or even 40 hours/wk). In 2011, we invested heavily in our next product, a new office, a lot of lawyering & tax accountanting (and made a couple rather big (hiring) mistakes with money), so our take-home wasn't glorious, but next year it will be.
We:
* built the product in a hack day a week or so for 3 mos * worked pretty hard the first year, since we were doing it alongside consulting; but literally there were weeks, and months, where we did nothing, and our business survived & even grew for a time * quit consulting in 2010 (me in Jan, my husband later)
To put this in perspective:
In late 2009, I had mono, which knocked me out for a few mos. I got better for a while, but then a series of viral infections landed me with chronic fatigue syndrome. Several times since then, I've been incapacitated for 1-3 mos.
Now I'm mostly better, but every time I get sick, I'm unable to work for a week or two. I don't make it up by working long hours when I'm well, either. We're talking 20-30 hours a week for me, max. We haven't worked more than a 40-hour work week for more than two weeks at a time in nearly 2 years, and our biz is 3 years old.
In short: it's totally doable.
Focus on solving a pain people will pay for. Build something simple, which doesn't require a lot of maintenance. Choose customers who aren't needy fuckers. Pick your hours.
Example: Our 1st product is a time tracking tool. Unsexy but it's got a run rate of just over $300k/yr now, just over 3 years old. We had 8% growth in revenue month over month from Dec to Jan. Which is totally sexy.
And… right now we're on a month-long trip to NZ. I've been working an average of an hour a day because I'm running a course to teach people how to build this kind of biz. (That's in 2-4 hour stretches, not actually 1 hour a day, which would be useless.) My husband hasn't worked at all on this trip except yesterday he had to reset one of our (redundant) servers because of an apparent hardware failure. That'll probably take a couple more hours of his time this week.
Sure, we could work more, hit it harder, and probably earn more money. But we're nicely on track for $1.5m in yearly revenue inside of ~2 more years (when I turn 30!) so I figure, why bother? What could we NOT have with $750k/yr of sustainable household income, even if we spent half the gross revenue on overhead (which is ridiculously unlikely)? We already get to touch thousands of people's lives every day. We can already invest in charities and nice things.
As Felix Dennis, who's ludicrously rich (in my mind), wrote in his very excellent How To Get Rich:
"Ask me what I will give you if you could wave a magic wand and give me my youth back. The answer would be everything I own and everything I will ever own."
He says no less than 3 times in his book that if he could go back to his youth and live a different life, he would have spent way, way, way less time accumulating his obscene (hundreds of $mils) riches and more time living. I take him at his word.
In short, it's totally possible. You won't believe it if you spend too much time reading HN, however. Or people will tell you that your ambition is too small… and sneer that you run a "lifestyle business." But then, you will have a life, so you can laugh all the way to the bank.
NB: I blog about this stuff at http://unicornfree.com in case you're interested.