I've read articles that say that when they stress-test fighter pilots by having them do cognitive test while wearing headphones with different sounds, they usually mix in infants crying. It's one of the most distracting things for a person to hear.

For me, it wasn't when the kids started crying that was the most stressful, but when they wouldn't stop. You check their diapers, you check if they're hungry, you rock them gently, and even then sometimes they just wouldn't stop crying. After a while the noise would be so bad that I had to hand them off to my wife, and she did the same when she became too frustrated.

Funnily enough, it made me immune to other kids crying. Before I had kids I would get very irritated at noisy kids or crying infants in the subway. Now, I'm just happy it's not my kids and that it's someone else's problem!