If you find a population that largely lives somewhere where drugs don't or can't grow, the results should be obvious once they do become available. Whether through a "culture of drugs and violence" or a "culture of drugs and love" or mere "natural market realities."
And in both cases, it's not as if we created drugs or smallpox with the specific intent of eradicating entire native societies. Or that our cavalier attitudes were at all in service of that goal.