The leap day system handles the mean, the leap seconds handle the variance around the mean. The need for leap seconds is not predictable—they zero out accumulated error.
We can imagine a system where leap days are split into mean and variance: This would look like a council coming together every thousand years to decide if that year will have a leap day or not, but otherwise we follow the pattern.
We can also imagine a system where leap seconds are split into mean and variance: Many years from now when the Earth is notably slower, there's a guaranteed leap second every odd month, and sometimes there's an extra leap second in June.