Stick your forefinger in your ear, well enough to form a bit of a seal. Then move your finger around a little bit. That causes a ton of rumbling/thumping noise as the air in the ear canal moves much more dramatically than it does when encountering sound waves in open air.

The same effect happens when you jostle a sealed in-ear monitor. I don't doubt that you are not perceiving this rumble, but my explanation is that your brain is filtering it out.

EDIT: It occurs to me now that if you listen at consistently high volume (which many in-ear headphone users do), jostling rumble will be relatively lower.