As a manager of a large organization, I have more respect for people on my team who say no and assert stronger ownership over their time, ideally by being transparent about their priorities. At the end of the day, we just look at impact and managing ones own time is a good sign that someone is focused on impact. Still, I'm sure that isn't the case with every manager in every company (or even every manager in my company)
The open space aspect of the FB offices is usually navigated by people putting headphones on to indicate they are heads down. I do it often even though I'm rarely listening to anything. I agree it isn't for everyone though.