Let me be clear -- nobody finds planning poker or story estimation fun. The same way nobody finds writing tests or documentation fun. So of course it'll be a breath of fresh air if it's not part of your job.

But the fact remains that in most environments, it's extremely useful for medium-term planning and especially in surfacing high-risk features that can send people down the wrong path for a long time. And it's meant to benefit stakeholders -- the people paying the developers' salaries -- not individual developers. It's about de-risking.

And if you really have the kind of team you seem to describe where everybody only works on their specific type of task and can't even estimate anybody else's work, then that's a danger situation when they leave or get sick or whatever and nobody else can step in and everyone's blocked because the only person who can do X is gone. Usually, you should be actively getting other developers to work on these features (usually the easier/smaller ones) and involving them in the estimation.