With many drugs that are used both therapeutically and recreationally, it is the case that the average recreational dosage is much larger. Ketamine is an exception here, as therapeutic doses are actually quite high. The common mistake here is to equate intravenous dosage with intranasal dosage, when the bioavailability differs significantly between these routes of administration. And that's not even considering that most reported recreational dosages are wrong due to cutting agents.

There certainly is recreational abuse with very large dosages, but I don't think it's fair to claim that the majority of users fall into this category.

Is it not assumed all recreational drugs cause or have the chance to cause damage to the body?

I always assumed it was a trade off of some temporary fun for x% damage to the body or x% increased chance of disease.

The majority? I doubt that claim.