I am a musician (singer, mostly) and when it comes to sight reading as a singer I would estimate that I'm in the 95th percentile or so. But I've tinkered with the piano recreationally for basically my whole life, at times more seriously than others, and yet I still cannot sight read on the piano for my life. There's just too much information for my brain to process in real-time. I've played some pretty difficult music on the organ, but I have to basically memorize the piece to get to that level.
I strongly believe that no amount of time at the piano will make me a good sight reader. I believe I could get marginally better, but I will never reach the level of even an average pianist.
Now one could be tempted to say: "you've spent far more time sight reading as a singer, which explains why you're better at it." And while this is somewhat true, it gets the causation backwards. It was obvious to me from a very early age that I was better at sight reading as a singer, so I spent more time doing it.
I sometimes wonder if this is where the 10,000 hour rule comes from: no one could stand spending 10,000 hours doing something they truly suck at. But on the other hand, I know plenty of amateur musicians who love what they do and have spent their entire lives making music, but will never reach a professional level (which is totally fine, but contradicts the 10,000 hour thesis).