The risk of psychedelic drugs is more comparable to motorsports, the more involved action sports, heavy equipment operation, etc. and your average person would be very wary of participating in these even though there are plenty of risk mitigations for these activities.
I'm not saying you shouldn't do drugs. I'm saying that many of them require way more respect than most people realize they need to give them, and trying to downplay that is not good. I can tell you from personal experience that it's easy to get yourself and people you care about in dangerous places.