Do you listen mostly to music with lyrics or accoustic music? I find the latter helpful but the former quite unhelpful - like trying to do math when somebody next to you is shouting randon numbers. But I have a faint recollection of having read research on this where it depended if you were a visual/verbal/etc thinker, and you were more vulnerable to disruptions occurring within your given thinking method.