Virtualmin is my second Open Source based business, and I'm pretty enthusiastic about Open Source in business. But the fact is that it's more about you than about open source or proprietary software, and nobody can figure out where you stand on this stuff but you.
I'll tell you some of the negatives first:
Open Source users are bitchy. All the time. They (or, I guess I should say, "we", as I was an Open Source user and developer before starting a business based on it), paradoxically perhaps, feel entitled far more than users of proprietary software. I suppose it is the long history of abuse in the proprietary world that has trained their software users to never expect help from their vendor, unless they pay a lot for it. Users of Open Source software expect bugfixes and answers immediately, if not sooner. They are simply far more demanding, and it's much harder to weed out your toxic "customers", since you can't stop selling to them. Since I'm a believer in weeding out your most painful customers, this is a big problem, in my opinion.
You must have longer range plans with an Open Source based business. The pool of potential acquirers seems to be smaller, as even after the success of several well-known Open Source based businesses, it is still considered riskier than proprietary software. So, plan for a long haul and prepare to be able to show a HUGE userbase if you ever do want to get acquired or raise money...but the next few items make this easier.
And now a positive or two or three:
Open Source is easier to "sell", so growing your userbase is remarkably easier than with proprietary software. It takes free to the next level and it's not at all uncommon to see an Open Source project with millions of users within a couple of years of starting. Webmin, our Open Source core, is downloaded over 2 million times per year, just from SourceForge. That's a really nice pool of potential customers, and the kind of numbers that make investors and acquirers swoon.
Getting your users to talk about you is easier. Open Source is inclusive, and a big pre-requisite for encouraging evangelism from your users is for them to have some ownership in it. Nothing makes a user feel more involved than submitting a patch, contributing to the documentation, answering questions, etc. Some of these things are possible with any software, of course, and many companies manage to create a feeling of ownership without being open...but it's easier with Open Source. Open Source users are far more likely to be fanatics about your software, and far less likely to be critical of its flaws (to outsiders, but see the first point for how they behave within the community...it's a two-sided thing).
Balancing the business needs of a company and the needs of an Open Source community can also be challenging, and in ways that you probably don't anticipate if you haven't been involved in Open Source development on a few projects. Pretty much my whole development career I've been drawn to OSS projects and development...so I don't really know any other way to do things. But it does have its drawbacks.
I wrote an article about this very subject a while back on the Webmin blog, which might be worth reading:
http://inthebox.webmin.com/open-source-and-business-a-precar...
I'm sure I've also posted a number of times on the subject here at news.yc. With about ten years of making my living off of Open Source software, I'm guessing I could be called a resident expert on the topic.