Oh yes it does. I probably caused measurable damage to my hearing trying to block out the conversations with headphones+music. It was horrific.
While playing music on headphones can help block out conversations, it is still a distraction. It's just not as annoying as hearing other people talk.
I've worked in places where it was an issue. And it's a very difficult one to address. "Please be quieter" is much more socially acceptable than "Please use deodorant".
1. In private, tell the person "I don't think your deodorant is working well enough. Just thought you ought to know."
2. Go to their manager, and have the manager say pretty much the same thing, not mentioning that anyone but the manager noticed anything.
Almost anything else comes off as boorish.
An open room needs plenty of space.
I don't mind if it's done at moderate volume and is relevant to the work at hand. Indeed, I like those kinds of conversation to happen around me; the ambient information lets me have a better feel for how the project is going.
It's only when it's irrelevant or loud that I'd want headphones. So we just kick that kind of conversation out of the team room.