I'm also skeptical how much value is added face-to-face. I think the long term strategy would be to apply big data techniques to hiring. If you do long-term tracking of hiring outcomes, you could start to develop a suite of accurate hiring predictors. It is clear at least some current selectors are not very good (e.g. gender, first name). I'm aware of an elite academic dept. that doesn't do face-to-face meetings for hiring assistant professors. Someone in the dept. claimed the application packet was the best predictor of the kinds of success the dept. hires for and face-to-face wasn't useful, if not misleading.
On the one hand, face-to-face does allow all these preconceptions and prejudices to happen. On the other, it allows the team to assess those things that actually matter in meetings, presentations, and other face-to-face venues.
Perhaps for a purely distributed company, this wouldn't matter. But in many existing cases, the face-to-face is still a factor.
As far as I know it works fairly well.
I still think you're not learning what you think you're learning from face-to-face. I don't have data on that, but then again, I'm guessing you don't either.