1.4 to 1.5 is the really painful upgrade. The rest are much less so. Especially after 1.6.
But you are right. At my previous job, I spent countless hours getting from 1.4 to 1.5, but the step from 1.5 to 1.6 was relatively painless.
1.7 was a very important release in the history of Django, just not a fun release to move to if you had a large codebase.