It might be similar to (albiet the oposite of) how putting on a pair of decent active noise cancelling headphones "sound different" (even if you were in an otherwise quiet room) and/or many people report a "head pressure" feeling.

Their brain is used to filtering out the background noise, so removing it causes strange sensations.