While I totally agree with not disrupting working people - I don't think you should ring fence them out of input in the process of defining product - we've all seen what happens in that case - "requirements" nailed down by a non technical team. And it's even more wasteful to have people working on the wrong things, or worst still being given so overly defined work that there is no room for them to add any value or find a better and faster solution.
While good management can help shield teams from unwanted intrusion - everyone needs to get involved in the process of manufacturing great software - which doesn't just mean attending meetings, but taking time to actively plan and consider the issue before the session. If your company values your input enough to bare the cost of asking you to be involved, then you should look upon that as a positive thing (unless they REALLY are just wasting your time with 'busy' meetings, which are really just wasting everyone's time - and it's a sign of either having money to burn with limited direction or a company going the wrong way fast)