You've got big moneyed interests who benefit from long IP terms and small groups of people that might benefit from shorter IP periods. End result, no legislator will push for shorter terms, they just don't care about it.
Disney can show how much money they'll lose if the mouse becomes public domain (and how much that'd hurt campaign contributions) while Mickey's auto repair shop will struggle to show similar losses if they can't put the mouse on their billboard.
You could run a piracy website that only offered works more than X years old, where X is some popular proposal for a reasonable term, and then fight a test case. To my mind the best legal argument against the current regime is that the Constitution's specification of 'limited time' copyright has been extended to be longer than the median human lifetime, thus making it effectively perpetual for most humans in the US; this also hinders the formation of a constituency for shorter periods because fewer and fewer people have had the experience of seeing a work they admired at the time of publication subsequently enter the public domain.
But realistically, any test case is likely to fail because higher courts would probably just say 'the term might be effectively infinite for people but it's short in the lifetime of a nation and corporations are arguably immortal so no media giveaway for you peons.'
How would you pitch a fifty year copyright to a random neighbor? "Tired of spending full price on Beatles albums? Wish you could hear more Elvis covers? Reduce copyright!"