The issue with alcohol is that it's present in a lot of foods in concentrations suffient to cause an effect. Same goes for fermented fruit. Additionally even today it serves as a preservative and solvent. You cannot meaningfully eliminate it from use.

Meanwhile drugs were always understood as something produced for the specific purpose of getting high.

There's a distinction because these are in fact different things.