On one team I worked in, we hired an older coder with bad knees and a couple of kids at home. Everyone in the team got along with him and generally liked him and he got his work done without too much fuss. But it was obvious that he didn't fit in with everyone else. We'd all hang out with each other socially (bars, clubs, weekend activities) and he opted out of those activities entirely and went home to his family every day at 5pm. And we saw other teams doing all these cool team activities (canyoneering, zip-lining, ski trips, etc) while we were limited to activities that were less physical and involved less alcohol. Over time, almost everyone in the team looked for opportunities to transfer to other teams (and sometimes companies) that looked like they were having more fun. Eventually, the team's output suffered as turnover increased, until eventually, he ended up leaving. The net effect was hugely disruptive to the company.
I can absolutely appreciate the argument that these culture fit tests become a way to discriminate against minorities, genders and people of a different age...and that's absolutely wrong. But we also need to accept that employees don't view work as a 9-5 working sentence that's completely separate from their life outside of work. To the extent that employees want work to be part of their overall lives, you need to hire a cohesive group. It's a hard problem to solve simply because humans are social animals, there's no universally correct answer and so much of our psychology isn't well understood by so many of us.