If it's a network issue, it's likely unsalvageable because the network is just crap. We need to be using whatever those are using. It is usually crystal clear and people are easily understandable even during large gatherings (EVE Online battles).
I'm the same as the OP, and what's worse, people talk 20 feet from the cisco conference phones (because those are everywhere) and it makes everything sound completely terrible. I don't listen half the time. That is definitely a microphone issue.