Not always, but how distracting is it really?

Obviously, everyone will be different but I get 2-3 emails from recruiters on LinkedIn per week. Managing them takes me about one minute per week.

Compared to the number of times where I ended up on a phone call or having lunch/coffee with a CTO, VP of Eng, architect or senior engineer, it's totally worth it to me.

My suggestion: don't cut out that option until it really becomes unmanageable. Chances are you'll find it's worth the occasional disturbance.