it's a tiny little more complicated than that.
The key element to grasp is that for all units above "second" aren't fixed: the duration of a minute, an hour, a day or a month or a year varies, depending on when it starts.
It's complicated by the fact that colloquial expectation - and likely often legally binding - is that "one day from now" is tomorrow at the same wall clock time.
So just expressing time deltas as seconds doesn't cut it, either.
This is on top of pesky details such as leap year rules (if multiple of 4 but not of hundred unless multiple of 400), half-hour time zones, and DST decided anually by parliament.
TAI and GPS - even if available - don't help you there because these things affect the algorithms and - worse - the data you need to store.
If you have a local time and forgot to store the UTC offset, you don't have a time. If you have a duration of one day, and haven't stored when this day starts, you don't know how many seconds in that day.
The particular problems of leap seconds are two: first, they are easily forgotten - leading to incorrect calculations. Second (heh!), since insertion of leap seconds is not regular, you need updates to keep your time arithmetics correct.
---
Now it might surprise you that I'd personally think leap seconds are cool, and we should keep them. However, it's hard to explain to someone faced with these problems why a few seconds of offset in sunrise time is such a big deal.
If you use time() function as POSIX defines it you already now get the "sane" number of seconds for the common calculations: to get the day after and the same time just always add 86400 seconds. It was so even at the day the last leap second happened (June 30, 2015). So the commonly used algorithms don't have to care. We already use the time as the astronomical time, add 86400 seconds and get the expected time a day after, and that there were 9e9 more oscillations of the Ce 133 atom between these two datetimes compared to other days we don't have to care.
Only if you change your system not to be POSIX compliant the calculations aren't obvious. Just don't do it.
The system as it is now, with the leap seconds happening, works. Who needs to care about TAI already cares. No need changing it.
http://www.wired.com/2015/01/torvalds_leapsecond/
"You’d have to teach people to not care about the correlation of time to the sun, or the correlation of dates to the seasons. That doesn’t sound very realistic.
The people and computers who really care about “atomic” time tend to be astronomers. For the rest of us – both people and computers – we’re probably better off taking the POSIX approach to it, and just say “who cares”, with a few unlucky people worrying about bugs happening because of the perversities of timekeeping."
Another way to fix the issue would be to define predictable leap seconds, just like leap years are predictable.
But I'd like to see them go. Nobody looks up to the sky and says "the sun is getting later and later". Half of the year most countries are far from solar time anyhow.
Do I care where the sun is relative to some arbitrary line in England? No.
Note that moving 30 km (or 20 miles) east or west also changes the sunset by one minute.
Depending on how close to the poles / equator you are, and probably also time of the year.
Of course a photographer would start well before sunset and continue well after, but if you want to program a machine to photograph su sets you will end up having to use techniques other than checking the time.