For the majority of people solar noon does not match civil noon. Today solar noon at my location was at 12:27.

If you live at the equator walking 500 metres east or west will put you a second off solar noon. That distance is reduced the further you get away from the equator.

Abolishing leap seconds would be unnoticeable for the majority of the population for thousands of years. At that point it could be fixed in one go or timezones could be slowly moved.

Those that would notice the change probably need sub-second accuracy that leap seconds just make more complicated.