While I agree that side quests usually are waste, what is the purpose of your playing? Is it because it's fun or because you have to finish? One could argue that the main quest is also a form of side quest and only for losers without better things to do...
If you go for finishing side-quests, you may not feel satisfied until you've finished them all. That can take considerably longer and be considerably less worth-while than the main game. (e.g. saving princess and defeating nemesis vs. collect all 500 random blue hidden items. Both fun, one takes far less time, and is more "worthwhile".)
Of course, that's quite a different thing to timesinks like "find all 400 hidden widgets" which can get pretty tedious...
Some games explain story elements in the side quests, for example. Others add additional gameplay experiences. When I buy a game, I buy it to enjoy it, not so I can ignore 50-75% of it, and as long as I'm enjoying the game (rather than playing purely because I feel I'm missing out on something, even though its not enjoyable), I want to get the most out of it. Especially if it was an expensive game (though I normally only buy games on offer).
Besides, most games have terrible endings, so its more about the journey (the quest, the side quests) than the destination (ending credits).
Its all about fun and enjoyment. Plying a game when you don't enjoy it is stupid. Also, if you have X amount of time to play games, then playing for longer than X is obviously bad, regardless of if you enjoy it or not, but the same is true for watching TV, playing sports etc etc. Everything in life must be balanced.