I've had two bad experiences with open plan offices, two good, and one excellent.
The first bad experience was shortlived - I was temporarily in a large room with half-height desk dividers with people doing sales and support calls. I asked to move.
In the other bad experience, it wasn't sales, but a bunch of reasonably unruly, not young, but childish developers (edit: could have been a culture/value mismatch, the company was a "corporate" supporting a long-lived product, my team coming from start-up environments or preferring them working on a new product). I managed to get my team moved to another location in the building, and a room for three of us to sit in (facing outward). (I stole the window seat.)
The first good experience with open plan was where our group was together, with a small gap between our group and the next, with some head-height dividers.
The second good experience was, I admit, not really open plan. In a long space, we had pods of 8-12 people with full-height glass walls, with frosting up to just above sitting-head-height. This was an interesting environment - we mixed support staff (non-phone support), operations staff, developers, designers, and so forth equally between the pods. (This was at Yola. Before we moved to new offices, we were in rooms in a house holding 5-7 people in each room.)
The best experience has been at Facebook, where I am here. Especially at our new campus in Menlo Park. Basically, open plan broken up with small "lounges" and curtain-closable rooms, with gaps between groups working on different things. Plenty of meeting room space, with tons of 2-4 person unreservable rooms, and whiteboards everywhere.
There are exceptions. Like in the old office, there are a few people who need serious quiet/lack of distractions to work, and they work in their own rooms. Some teams go into "war rooms" when they're near release to facilitate even higher-bandwidth discussions.
This applies to the bulk of the company, including, for example, the CEO, CTO, VP of Engineering, and VP of Product.
Anecdotally, open plan doesn't work at all well when you mix certain job types (sales and engineering, for example), doesn't work as well when you mix people working on different products or without break-out rooms or gaps between groups. It seems like it can be done well, and it probably needs motivated employees (employees already unhappy are likely to respond more aggressively to distraction). Some people just don't work well in the environment, and catering to them is a good idea. Having meeting rooms or collaboration areas located close by, but not distracting to those working, makes people tend to use them to avoid annoying those nearby (have this happen every few days when conversations go over a few minutes, or have more than a few people).