Hence you can’t make the change incrementally, which is a big risk.
Concerning Linux distributions, I seriously doubt if they have resources to assess the performance impact across the wide range of software.
Concerning performance impact, we end up with more instructions and more memory accesses and increased register pressure. This has a chance to make things slower.