>Perhaps focus on rapport at the start of your interviews?
I don't think that is the issue. I was looking for an entry level position at the time, it's a CROWED field, and the handful of places where I and another much younger classmate both interviewed where I could compare included at least one, often more phone interviews. If I wasn't getting along well with them I don't expect they would have brought me in. Most of the phone calls and interpersonal in person interviews went great IMO.
To be clear I have no idea what their idea of "culture fit" actually was.
I've since started asking, nobody has been able to answer without first looking confused and humming and hawing a while. Most of the answers were usually surprisingly generic and such about "openness", "fun" and such, I suspect a few were made up on the spot. Amusing ;)