The brain can push sounds to the background in the same way it does with visual cues, if there's no danger associated with them. For instance, you're not annoyed by the sounds your keyboard makes when you type (and I know many developers, who prefer loud, mechanical keyboards).
Also, try the following experiment: open a window during the day, plug in a microphone and just record with it for a minute. When you play it back, you'll hear many sounds your brain learned to ignore - cars passing by, hum of your computer, kids playing soccer on a field nearby, etc.
For those reasons I think there might be a set of sounds that would not become annoying with time.