You don't need to pick all of your managers, coworkers, and clients. That's the point of vetting a company culture before you join. In general, the culture should be a reflection of a shared ideology for business and interpersonal behavior. There is a ton of room for variety within that, but it pins some underlying important traits.

Being a good culture fit does not mean you have to like them so much that you're inviting them out for a beer. And in order to really be a long term competitive company and team, you need to be able to be unified first. If there is significant internal turmoil, or people are unmotivated, uncomfortable, threatened, etc. you will never be competitive. "Building character" is generally code for "deal with this crappy situation", and life is too short to make people miserable. You can get a diverse group of people without making all of them unhappy if you select for people who can work together.