I agree with that line, but it feels like sometimes we are tricked into feeling like we're enjoying something, when we really are not. What matters is if afterwards, if you feel like you wasted that time.

I was about to correct you and say that was Bertrand Russell but apparently it was neither of them - http://quoteinvestigator.com/2010/06/11/time-you-enjoy/.