Both are true. When you're bored you get into dumber kinds of fun.
I've smoked marijuana before. I've been drunk. I haven't had sex, because I'm into the idea of abstinence, but I'd done everything but. And absolutely those things are enjoyable.
The difference is that I don't rely on stuff like that. I write, I program, I design things, I read, I try to write music and shoot films. I don't feel the need to drink once a week, or to go out and randomly hook up. And that's because I have so many things I can do at any given moment.
If you don't have that stuff, you're forced to rely on sex and drugs. And those things are both easily abused.
Therefore, it's "dumb" in that it requires no effort whatsoever on your part. And if you come to rely on that stuff for entertainment, then you're not gaining anything as you go. As opposed to its being used sparingly, in which case you're off doing more productive things.