Didn't we give up on having "the time" correspond to solar time in the 1800s with the advent of railways? Unless you live exactly upon a specific set of lines on Earth, 00:00 never corresponds to midnight, leap seconds or not.
On the other hand, we already have leap seconds. They're not much of a problem.
Well, leap seconds are going away, so we don't.
They were never adequate for the long-term anyway, as the "two leap seconds a year" scheme will break down as the earth's rotation slows over the centuries and more and more leap seconds will be needed.
We're talking about an hour time drift over a few thousand years. Does that really matter? It's like caring what Julius Caesar's sense of "morning" or "evening" was. They never really solved any practical problem.