Imagine you have a purchase form that converts at 0.1%. You test a new change, and with the first 22 users (11 in each group), if the control group randomly gets a conversion, it will have AT LEAST a ~9% CVR now. Okay, maybe you ignore that data, since you can just compare to historical data. Whatever.
Even if the new form is dramatically better and has a 30% CVR -- There's a 3% chance it would show a ~9% CVR also, and about the same that it would show a 0% CVR. Similarly, there's about a 6% chance it would show a 91%+ CVR -- skewing your results just as badly in the opposite direction. There's only like a 30% chance you get close to the "true" CVR.
Unless these hydroxycloroquine is close to 100% effective (and that doesn't seem to be what any of these studies say) I just don't understand how there's any confidence with small sample sizes.
Does anyone have a link to how this works? I've asked a few friends in medicine, and they haven't given a good answer. Haven't found anything on Wikipedia. Probably don't know what to search for.