Most of the side-effects aren't severe. The worst cited are usually blindness and deadly heart arrhythmia. The problem is when you start giving it out to a lot of people, you start to see those low-incidence side-effects increasingly often.
It is a basic and important medicine. It's not that there are no concerns about its safety; it's that the safety concerns are well-known. Those are distinctly different statements.
(As an aside, for example, Tylenol and aspirin can both be quite harmful, but under what conditions they're dangerous is well-known so they are widely used pretty safely. Even so, you still find people inadvertently destroying their livers with Tylenol/paracetemol when they don't realize two OTC medicines contain them and overdose).
So then the question is how much harm are you doing by giving people the drug (under some set of circumstances) vs how great a benefit might someone derive from it.
Clearly on one side, if you gave it to everyone as a prophylactic, that'd likely cause more harm than good. Perhaps, you give it to people who have tested positive but are not admitted to the hospital. Or you can give it to only those who are admitted. Or you can give it only to people who are doing poorly.
Presuming the harmfulness of the medicine is well known you can run some easy hypothetical numbers about how many people will have serious harm or death from receiving it on a wide scale. We have a rough idea of the range of people who will die or have serious cases of COVID. The only thing left is the possible benefit people may get from the medicine.
As of now there is no indication that the benefit is worth the harm to give it out to people in a wide-scale fashion. Additionally, the current studies don't even support giving it to people who are in-patients. In most studies, patients receiving it fare worse than those not.