Leap years are regular, leap seconds are not. Skipping leap day would be earlier and more obviously noticable than skipping leap seconds.