I guess the "efficiency" gains of remote (more focus time for individuals? less distractions?) are more continuous/less thresholded OTOH because they're more related to individuals than the collective.
But basically the top reason (if true) means you kind of need to be all in on the in-person or you're kind of all out efficiency-wise. Very small companies I guess can go either way and be OK (though, people need to be extra passionate about their project and coworkers to be motivated without that in-person connectedness).
Big companies I think just don't work at all in the remote world. Everyone loses meaning. No one wants to do those big audacious or really challenging/messy things with no meaning (I mean, people will still do a lot of small things to get their paycheck). This may sound bad but...I am absolutely not going to do a giant, messy, complex refactor for a bunch of zoom heads who I can only talk to through some awkward hand-raising-button-disconnected-digital-cubicle social dynamics.
It's just inhuman the world we were encouraged to be a part of. And we talk as if it's "normal"... Anyways, yeah efficiency! Yes, it's just inefficient for big companies. And eventually we'll realize this. I hope.