I’d love a special or grouping aspect so that you can limit or focus conversations.
I’ve been in a VR hangout and one person was monopolizing the conversation. Walking away is a very natural signal and allows for a new side conversation to form.
It doesn’t have to be VR of course, but some kind of break out session while the main content continues.