Lots of people come to my team asking for work on X. If I said yes to everything my team would collapse from being spread too thinly. Being able to say that X is lower priority than what I am currently working on and leave the door open for re-evaluating priorities if needed is a key part of my job.

My team also frequently comes to me with ideas for interesting work that also isn't contributing to our vision. If we just did everything that came to mind we'd also fall apart.