With anything else (books or stories, pictures or movies, etc) the ability to modify or extend the work was the default. Copyright was a carve-out in this.
With software it's actually the reverse--the ability to modify or extend the work is _not_ the default. It takes explicit action by the creator to make that reasonable without substantial effort in most cases. We're actually dealing with an entirely different situation here, and providing that exclusivity on top really does seem like a bad deal for society in a lot of ways.
Is there anything else that's covered by copyright that's in a similar sort of situation as software? Where the thing that's covered by copyright _isn't_ really modifiable to begin with?
Which is a lot of words to say--on the surface, yeah, I agree with you. Besides shorter terms, I think if you want that exclusivity from society you should be required to give something back in return... like the source code so everyone can benefit from and build off of your work after your period of exclusivity expires.