It's no different than using Skype or Ventrilo or even a phone/conference call. There's no real way to manage who talks to who while you're playing a game.