They sell them without wires
The same effect happens when you jostle a sealed in-ear monitor. I don't doubt that you are not perceiving this rumble, but my explanation is that your brain is filtering it out.
EDIT: It occurs to me now that if you listen at consistently high volume (which many in-ear headphone users do), jostling rumble will be relatively lower.