Time. Neither of us can afford to quit our "jobs at a big company". We're not 20 and have mortgages, school payments, etc. Working part-time is probably 10 times less effective. Ohh... I cannot stress it enough! Part-time can be as bad as 50 times less effective; sometimes weeks can pass with no work being done, depends on how busy we get at our day jobs.
Marketing&Advertising. We cannot afford a solid PR campaign and expensive ads. We got some independent evaluations with ad quotes as high as $500K/year. Google brings a steady (albeit very small) stream of daily new users, but unless we come up with something, it will take forever to reach good numbers. Common advise like "blogging" BS and "sign all your emails with your URL" do not apply and really make no sense - you can't get into 6-figure user numbers this way.