The problem really lies in the way the IRS limit works. It limits the amount of shares you can purchase based on the FMV of the stock on the offering date. So if your stock goes down then you may well hit the limit and not realize it.
But you are correct, the 25k limit is based on all of the purchases made in the calendar year. And the $25k carryover complicates things as well.