It surprised me when I moved to the US from Canada how many elderly planned to stay in their homes. In Canada it felt like many people downsized to smaller condos, at least in the city.
I get the complaint that people were being taxed out of their homes and that's not good, but in the end it just shifts the tax burden from the old to the young.
If the main reason somebody pays less taxes is that they bought the house before you were born, that feels like age discrimination.
It's actually theft by corporate (or family chain) long-term property owners from everybody else; it doesn't favor individuals of particular ages (especially since the parent-child residence exception was adopted.)
I'm not elderly, but that's a few reasons why they wouldn't downsize.