Very true. Some sounds are worse than others, depending on the degree to which their frequency distribution differs from that of the environment. Thus, construction noise (jackhammers and so on) is annoying, but you can get used to it. The warning beeps on reversing vehicles, on the other hand, drive me towards suicidal despair, and no, I am not exaggerating for effect. I have never seen any actual evidence that they reduce accidents; they seem like a good idea that was adopted to reduce liability without any evidence of actual effectiveness.
Other unnecessary but infuriating sources of noise pollution are car alarms that don't know when to quit, and leafblowers, which are not only extremely noisy due to the 2-stroke engine, they pollute heavily with fumes (very unhealthy for the operator) and typically just move dirt and debris off private property and into common areas, exploiting three externalities for a very small labor saving. I must confess that I fantasize about meeting the person who invented them and exacting a horrible revenge for the intense distress they cause me - again, I'm not exaggerating for comic effect. If someone starts one of those up within 30 feet of me it takes all my strength not to fall to my knees and start vomiting.
I liked living in cities when I was younger but these days I'm trying to figure out the logistics of living in a remote location like a forest or the Mojave desert, which is the only place I've ever felt truly relaxed. I worked there for a month (with a film crew) a few years ago and it was heaven.