It's no more distracting than talk radio or some blaring annoying commercial which doesn't stop talking if I need to focus. When you have a handsfree device, just say "Hold on" and do what you need to do. You can always ask them to repeat themselves I you don't hear.