I guess you develop some degree of trust (and a stronger mental model) during a meeting that persists afterwards.
I've been working remotely for several years. I changed jobs in March, right at the beginning of the COVID-19 pandemic really impacting daily life. As a result I've never met any of my colleagues in person. I often feel like I don't have a good feeling for how they feel about a given topic and spend far too much time and energy trying to manage those relationships.
I'm looking forward to being able to travel again so I can meet at least some of them. In the past, I've participated in "team building" events where the entire (small) engineering group rents out a beach house somewhere for a week. That time is spent split approximately evenly between regular work, brainstorming/planning for the future, and social engagement. I didn't see the value of it before experiencing it, but now I'm 100% in favor of it.
If I ever get to the point where I'm leading an engineering group at a funded startup, semi-annual engineering retreats will definitely be a priority for me. They don't have to be "rent a beach house for two weeks in the Outer Banks", either. I live in rural Arkansas, and there are large cabins on the Buffalo River and White River that go for ~$200 / night or less during the offseason and can comfortably sleep 35 people.
Also, granted, I live in this area and have personal contacts I can reach out to. The listed prices are going to be much higher than that - but last year I reached out to the owners of a couple of these places personally to see about booking something in the offseason, and that was the lowest price I got.
For someone not local and able to negotiate face-to-face, I'd say $500-$700 / night is about what I'd expect. Even so... when you consider the cost of travel and the cost of more "traditional" locations, the difference is pretty much insignificant.
For me, the former flows from the latter. When I meet in person, there is eventually a moment when I get a click feeling. Following that, a person's behavior becomes more predictable. Or at the very least explainable. That leads to comfort, which is a commercial substitute for trust.
That click take me a long time reach on the phone. For whatever reason, it has never happened over video chat.
When we met in person it was like we'd known each other for years because we had known each other for years.
I think the issue here could be related to the dissonance of seeing a face, the brain expecting eye contact, but being totally unable to get it because of technological limitations. Since the brain doesn't have any other "source of trust" for that face (be that the voice, past behavior/experiences, having met that person to being with) it clicks the "do not trust (entirely)" response.
Of course I'm not a psychologist/psychiatrist/neurological MD and I'm talking entirely from my experience.