There is nothing worse than hearing one side of phone conversations when I'm trying to concentrate; it's like having a power drill grinding into my temple.
It seems like a lot of management types love open-plan offices because of the opportunities for serendipitous communication. I understand that, and can see the value if you're a CEO and your job is to be on top of things.
But a programmer's job is to be on the bottom of things. Hearing about some change to some component through the office chatter is completely immaterial to me and my day unless I'm actually working on that specific piece of code at that moment. All "being on top of things" means is that I'm distracted from the stuff I need to focus on.
Would headphones w/ noise cancellation help? Even if you didn't play music through them?
KISS; sounds like some really expensive ear plugs.
When I need to concentrate, I wear my big Sennheiser headphones - often not even plugged into an audio source.
They are very comfortable for long periods, block out most of the noise, and (best of all) they send a clear "do not disturb" message to anyone thinking of interrupting.
It drives me fucking insane when I'm trying to work stuff out mentally to hear people speaking.
Edit: ear buds aren't great since I'd like to still hear in 10 years, plus, in my ears, they get uncomfortable around hour 4-5. And I've tried the full variety of foam and hard, medium, and soft plastic that Shure offer; they still get uncomfortable after long enough.