Because you're paying it off faster than it grows. It would be exponential if you were allowed to simply let it sit without paying it down, but that would break the contract and get you foreclosed.