You’re wrong and have the situation exactly backwards.
If a day has 86,401 or 86,399 seconds due to leap seconds, POSIX time still advances by exactly 86,400.
If you had a perfectly accurate stopwatch running since 1970-01-01 the number it shows now would be different from POSIX time.