That said, this is true of many other substances as well. Childhood obesity and alcohol abuse also cause developmental issues, and perhaps these should all be age-restricted (sale of sugary foods in the case of childhood obesity).
However, I would not forbid any adult the right to do what they want with their body as long as they do not infringe on the rights of others. If someone wants to ruin their life with drug abuse, while pitiable, it is their option. This has some negative externalities but, in my opinion, it is better than both government restrictions on personal freedom and the black market that has developed around illegal drugs.