Most active noise-canceling head-phones work pretty well even if you don't listen to anything.
Or, you know, tell people to be quiet.
I completely agree with you, without music, noise cancelling headphones are pretty much useless to help you focus by canceling 'asymmetric' noise.
My personal setup is to have a pair of earplugs in resina as a first layer, my headphones as a second layer, the noise cancelling option on and finally low volume music. With that setup I can't hear anything to the point where I get scared shitless if someone disturbs me. That's how deep in the zone I become!
Solution for this was proposed in Diamond Age by Neal Stephenson. Engineers were working in big room, but they were so focused, guests had to use special protocol: stand beside desk and wait quietly until you are noticed.