Cannabis has been a Schedule 1 drug for decades, which makes it, according to the DEA's own definition, a drug "with no currently accepted medical use and a high potential for abuse."
It's only in the past couple of years that there's been any sort of serious talk about potential health benefits. Marijuana use and sale was first banned in 1937, and when that ban was later declared unconstitutional, the Controlled Substances Act of 1970 created the drug scheduling system and placed it in the "most dangerous" category.
Next year will mark the 80th anniversary of the original 1937 ban, so it seems hard to claim that the prevailing "Marijuana talk" of our time has focused on the health benefits, rather than the dangers. The reason there's been a small but constant discourse on the health benefits of cannabis over the past eight decades is that the current laws and the official DEA stance are clearly overzealous.
It's as if soft drinks were outlawed, and our government declared that they were as bad for your health as heroin. In truth, if you are a sufficiently heavy "daily drinker" of sugar water, you will develop a severe chemical dependence, and eventually wind up with type 2 diabetes, which will permanently damage your health and certainly shorten your lifespan. However, there's nothing wrong with having a root beer every now and then.
So if soft drinks had been illegal for the past 80 years because they had "no currently accepted medical use and a high potential for abuse" (which is true), you'd definitely be hearing some determined voices pointing out that there were some potential health benefits of sugar and water.
This is the number one stereotype of marijuana users, but I implore you to not lump all of us into that group. There exists a large portion of users that respect intelligence and try to lead interesting lives.
A big problem is that cannabis can make you feel totally OK with whatever you are currently doing, when otherwise you may feel bored and become motivated to do something else. Mix this with the fact that regular marijuana use probably won't destroy your life (it's simply not a hard drug), and you get people that float through existence feeling fine but then one day realize years have gone by and they forgot to develop unique skills or an interesting personality.
However, many people figure this out before it's too late and make sure that getting high goes hand-in-hand with doing something interesting. i.e. smoke and then create art, music, or software. For the people who don't realize this, well... the cold truth is that many people aren't smart or talented to begin with and maybe need some alleviation from the more painful aspects of the human condition. I believe weed is a better solution for this than hard drugs, Rx pills, or everyone's favorite toxin: alcohol.
Last of all I want to say that smoke is clearly carcinogenic, but there have been recent studies suggesting things that people would have thought crazy only 10-20 years ago:
http://www.cancer.gov/about-cancer/treatment/cam/hp/cannabis...
The other reason being the gradually mounting paranoia eventually culminating in what I can only describe as a paranoid anxiety attack.
The day that happened I quit.
I'm pro-legalisation but I abused it and ruined it for myself. Many others do too, all the while espousing its complete lack of drawbacks, which I find irritating.
That said, I personally don't feel comfortable indulging in something chronically that has such pronounced effect on my mental abilities. Even if it has no long-term physical consequences, the habit (and what comes with it: junk food, junk tv, lethargy, etc.) itself can't be ideal.
Perhaps it would work if I didn't have access to the internet and/or lived in a more nature-like area. Or perhaps it would work if my lifestyle was already different. I don't know. But as it is, if I smoke I become either couch-locked or jittery and paranoid.
I think individual differences matter a great deal, and you're right in pointing that out.
That said, the vast majority of regular pot smokers I know are more similar to me than they are to you, so based on that I think the negative bias is at least somewhat justified. I'd never tell someone not to smoke, I wouldn't judge chronic smokers no matter their lifestyle. But I've warned every one of my siblings and those curious about it to keep an eye out for how they respond to the stuff. It's nowhere near as potentially dangerous as alcohol, I think, but it's also not quite like picking up a coffee habit.
(I do have to confirm that I would occasionally get a burst of 'creativity' in particular when coding on purely-fun side-projects! Doing so was enjoyable and funny albeit not always very productive. I miss that sometimes.)
You should re-phrase this as: "the people I know who publicly advertise the fact that they smoke pot".
In my experience, there are a lot of marijuana users who are otherwise extremely normal people (and even sometimes on the exceptional end of normal). But given the legal status of marijuana, an ambitious middle-aged professional with a young family who also smokes a couple times a month is unlikely to publicly advertise the fact.
Because you (presumably) don't smoke, those people are extremely unlikely to confide in you about this particular subject.
A good counter example is someone I know who didn't start until around age 25, only indulges occasionally (so it isn't an addiction), and has very high cognitive ability (currently making 6 figures in the financial industry).