From my experience, it is not so much the noise level, but the distractions that they create. A nice set of noise-cancelling headphones allow me to focus in on my tasks.
One thing I did notice was I have to pop my ears when I have the noise cancelling on. A few times it also made me feel nauseous when I turned them on.