> I began to suspect it was because of "culture fit" -- will this 35 year old introvert fit in at our frat house/office?

The job does not have to be a frat house office in order for a candidate to fail the culture fit. Unfortunately many programmers think programming is 100% writing code, when often times it is more about communication and relationships. If you were missing a certain technical skill what would you do? Now apply that same mindset to being a little less socially awkward. No one is saying you have to learn to be Dale Carnegie himself, but even just a small amount of people skills goes a long way.