Parent of 2. Babies are distracting by design:) they cannot talk and they can only get attention by crying/cooing, and we are wired to listen and prioritize the sound of babies crying/cooing over other activities to guarantee their survival:therefore if there's a baby crying/cooing at a conference the attendees will give priority to the baby over the speaker. I personally didn't think it was rude to ask to go strolling outside as this was from my perspective distracting for the participants.