It's a weird one and I really do think a lot of the time it depends on who's saying it and in what context, I think when there is a large power imbalance than the use of the word disabled can be really horrible but between peers, not so much.
Personally I prefer being called Stuart, possibly "you know, you've met him before, he's the one in the wheelchair", that sort of thing. I don't think there is one right answer because you obviously can't please everyone all the time, but I do think that large institutions and those with power have to be very careful about how they phrase things because you will come off as dismissive if you're not careful. Even if you don't need to be.
My company did some work for an organization that supported people with disabilities in the workforce. The client stressed the importance of the distinction between using "people with disabilities" as opposed to "disabled people," with the former being much preferred because - due to its structure - it puts the focus (i.e. the first word) on "people" rather than "disabled/disability".
When people can't read a website it's tempting to tell them to ctrl + or install user CSS. But better is to just design the site in ways that don't exclude large numbers of the population, let alone people with visual impairment. It's not that person's eyesight that excludes them, it's the designer's poor choice of font size and colour that excludes them.
There are problems with this model, but it's still mostly useful.
Of course there is a spectrum, and at some point nearly everyone is unable to do common tasks we usually take for granted. I'm amazed how quickly I forget after recovering from an injury or finding a workaround to a physical limitation.