The most annoying and least suspicious thing for me personally would be the person on the other end playing back my own audio with a delay, just like the echo of your own voice that you get sometimes on a bad phone connection.

This way I would blame it on my connection, not the other person, who is actually causing it. In a team setting, it goes like "Oh, so we're all hearing our own echoes. We should quit and try again" If there is one person acting as reflector with the right delay (echo generator) for everyone else, that could work, in theory.