There's some legit possibility of harm to people with genetic disposition to some specific mental illnesses like schizophrenia. It can bring in psychosis in people prone to it. Even though it can be very helpful to sufferers of other mental illnesses.

I've been warned about this by psychiatrists who were otherwise very liberal about drug use (one of them I knew even boasted to have tried almost everything). And it's legal in my country. Which I agree with, the potential harm for a small percentage of people doesn't mean it should be banned for everyone.