While not as physically safe as LSD, opiates have essentially no serious long-term health effects other than constipation. The major downside is the highly visible acute respiratory depression after overdose.
Edit: Speed is used by tons of people, and Paul Erdos said it gave him inspiration (though methamphetamine might have more downsides than other amphetamines, similar to how barbiturates are rather outclassed by benzodiazepines - but that's hardly reason to outlaw it). Many artists have used cocaine to their benefit. I'm all for psychedelics, but don't pretend that other medicines don't have plenty of value.