A tiny team (2-4 developers) with some sort of walls (so you aren't looking over your monitor at someone else) is probably a good compromise between getting things done (concentration) and "collaboration" i.e. keeping others from concentrating.
I concentrate really well but it's still hard for my brain not to react to hearing my boss asking another engineer in the next cubicle say "What's the kind of matrix where the inverse is also its transpose?" That actually happened to me once and I blurted out the answer. When he then asked me directly, "Hey, you know about matrix math?" I smiled and told him "Not as far as you know." :-)
In my home office, which is a couple of miles outside the city, all I hear is birds chirping and rustling tree leaves. I guest I'm a bit spoiled in that way but it's a great working environment.
In my opinon, open floor plans serve only two purposes: lowering the costs of building and maintaining an office and keeping the staff monitored.
> Facebook now has a 2800 person room in its headquarters [...]
That sounds like a complete nightmare to me.
I have yet to see someone that performs better when constantly distracted or interrupted. Socializing ? I do that in my free time at other places and with people that I choose on my own.
Employee monitoring ? If you need such desperate measure to ensure that workers actually do their work, there's something terribly wrong.
Best setup from my personal experience are small 2-4 person offices.
My productivity suffers when I am interrupted seemingly at random by the needs of those inconsiderate enough to assume their need trumps whatever I am working on at the time.
Email, texting and instant message are controllable mediums. The only acceptable barrier to the real world interruptions accompanying open concept is a pair of headphones and that is too light a defense in my experience.
For example there are thousands of HN users who work in companies with <20 people. I personally have more than 5x that number of people on my floor.
There are companies where anything not directly work related is discouraged and then there are companies where having a 20min coffee break with co-workers, or having weekly team events, etc is encouraged. In the first kind of company it would be isolating to work in a private office, whereas in the second kind of company it could work well.
Etc, etc. Personally I hate open offices... but I work next to sales people and a few very loud co-workers.