I do all of my personal and business banking at a local credit union. I like the fact that the shareholders of the credit union are the same as the customers. For investments, Vanguard also uses the same approach. This doesn't guarantee that it can't be poorly run, but at least then there is no conflict of interest as far as doing what is best for the business against doing what is best for the customer.

In general, the best advice that I have heard and read all points to going with a local bank or credit union. Unless you have millions of dollars, you're just a number to a large bank. With a local institution, you can develop relationships with the people in charge.