The most successful startups are disruptive. How can someone be disruptive when they're worried about keeping up appearances? To my mind, worrying about what others think of you starting a company might just mean you shouldn't do it at all.
Obviously there's a big difference between whether you are perceived as responsible and whether you really are.