How does this compare to Tailscale/Headscale?
https://tailscale.com/blog/how-tailscale-works#encrypted-tcp...
And Tailscale is doing a great job at trying to only resort to relaying when there's really no other choice, both because it improves latency and saves them bandwidth on their relays.
It's very similar to VoIP and videocalls (1:1; conferencing usually performs better with SFUs) in that regard.