I'm not sure why but I wouldn't feel comfortable letting some tech bros decide what drugs people should take.

Yes: but what was implicit in mine is that that output vector is no more or less informative than the current method of associating strong symptoms with specific diagnoses which have known drug dosages. Same idea, different representation.

The difference is whether the process can be automated, and that is where the problem of the black box comes in. With the current system, we know why the drug was prescribed. With an ML system, we would have no clue.