Also, super specific to type of disease or disability, but my arthritis is way worse if I get cold. My hands are practically useless for half an hour after if I don't properly bundle up.

I've got a mild case and normally don't even think about it except for the two puffs of medicine a day, but if it's -15C or below I really notice my asthma and have to adjust my breathing technique, like breathing through the mouth with my tongue raised to heat the air before it hits the throat.

If you got a severe case you can get special masks[1] which contains a heat magazine/exchange element, however it restricts airflow so if you already struggle with shortness of breath this might not work for you.

[1]: https://lunginstitute.com/blog/ct-masks-for-copd/

So yes, cold DOES affect people with disabilities more, because they are not exerting themselves when out and about the way able-bodied people exert themselves.

Now combine this with the points the others have raised.