I wonder if the reason we get happier b/c we get older is because our sexual drive decreases. As it decreases, we focus less on what we can't get (more attractive partner, more exciting sex, etc), because it simply doesn't become as desirable.
Interesting or maybe as we get older we have satisfied/achieved aspects of that sexual drive and no longer see it as the end all be all with respect to happiness. Either way I think you have a point with the narrative that sex = happiness for some.
My sex drive only increased after I chose to be happy.