I really miss the contact from the office but since most people don't go in any more, it is not worth much of my time to travel there.
Yes, we employ some people fully remotely, which is fine because that's what we knew at the time so we do get the benefit of the wider pool.
However, just because "I work really effectively at home", it doesn't mean that there are people who are not taking multiple trips to the shops, to school etc. since they can and although they could be doing their work when they get back, there is also a good chance that they won't or they'll be distracted etc.
Our productivity has definitely been lower during remote-only working although it is hard to attribute that to any one specific thing. The truth is that particularly in non-quantifiable work like development, it is hard to get a thermometer on how productive people are. The gut is unreliable and we want to trust people but if for no other reason, the office has less distractions for most people who can concentrate on their work more.
Could people make it work better? Yes. Will it always work well for everyone? No.