Hacker News Reader

Comment by person_of_color | Hacker News Reader

How likely is this due to random chance?

Well, given that 90/15000 = 0.6% in the control group developed the disease, you can consider the vaccine group as a Bernouili trial of n = 15000 with probability p = 0.6%. Then the probability of observing 5 or fewer cases is 3.4*10^-32, from the tail probability of the binomial distribution.

https://en.wikipedia.org/wiki/Bernoulli_trial

Of course, that's assuming that five guys from the vaccine group didn't get infected at the same after-ski party, or any funny business that violates statistical independence ...

Naive question: If the null hypothesis is that there is no difference, wouldn't that imply 95/30,000, p = 0.0031, putting the probability of observing less than 5 cases at a much more reasonable 1*10^(-14).

I think Fisher's exact test [1] is most commonly used in these types of trials. But the P-value roland (parent comment) provided also make sense to me. For the Fisher's test R says...

                 Covid    NoCovid
                 _____    _______
    Vaccine        5       1495  
    NoVaccine     90       1410  

    9.0e-22  one tailed
    4.5e-22  two tailed

[1] https://en.wikipedia.org/wiki/Fisher%27s_exact_test

Shouldn't the NoCovid values be 14995 and 14910?

Ah yeah good catch (2.7e-21 for 2-tailed, mutatis mutandis)

No, I think you must estimate p only using the cohort where people were not treated, otherwise you will underestimate the population fraction. In order to test if the null hypothesis is true, we can't assume that's its true when constructing the test.

A 90 to 5 by random chance?

Not at all likely

Reply on news.ycombinator.com