Yeah how ever will they live with having to wake up slightly earlier or later for their entire lives.
My understanding was that leap seconds are only ever added. It’s not like this averages out over time. The problem is that because they are only added the effect is cumulative and eventually some future generation will be eating lunch at 00:00:00.