I remember being a hard ass about ensuring that our office was full of life whenever a prospective client came to visit. Even though my employees could just as easily work from the corner coffee shop or home, the idea is: A company with overhead and payroll is obviously doing something right and likely more permanent than this freelancer I met working from his house. Therefore, they're more likely to succeed.
The perceived risk of failure is the #1 reason proposals are rejected. You don't need a brick & mortar office or whatever to help mitigate that risk, but it helps.
As far as overhead goes - currently it's mostly the same, but part of my motivation to go the "company" route is the ability to hire someone else in the near future. Once that starts to happen there will be a lot more overhead to consider regarding labor costs, employee income taxes, hardware, etc.