I am extremely affected by outdoors noise during sleep. Traffic, construction, kids crying, people shouting..
I try my best to convince myself "I'm perfectly comfortable, nothing's happening in my bedroom, whoever is causing that noise can not affect me in any way" but to no avail :(
If the source is indoors, like the whirr of a hard disk (very annoying on recent Macs + external hard disks, constantly spinning up and down) I have to turn it off before trying to sleep.
It gets so bad that even the mere anticipation of an expected noise, a clang or a honk or a yell, prevents me from falling asleep. For me it's definitely psychological than physical. Sort of an arrogant insistence that everyone and everything should be silent while I rest.
Thankfully when I'm really tired I stop caring about noise and just drop like a log.