well apathetic individuals still rely on the society they are in, most of the time anyway. I think this is where the narcissism perception comes in.

Most stoners in particular people who smoke a lot and for a long time at least in my experience become apathetic and to a degree dependent on their environment. They rely on others to organize their lives, even get simple chores done, they become sort of lethargic. And when you have an individual like that who also acts indifferently towards it you create (I think a justified) negative perception.