Everything you experience is bound to be mind-altering in one way or another. We change constantly, and our past selves from 10 years ago are hardly the same person as our current selves. (In fact, the very notion that there is a 'self' is more artificial and murky than anything.) Why experience anything then, if the thought of changing is so terrifying?
Of course, that does not mean you should experience everything. I would stay away from acid if I had known schizophrenic antecedents in my family. But that's like with anything: be aware of risk factors, act accordingly. Don't eat too much sugar if you are diabetic, don't explore caves if you are claustrophobic, and so on.