For First Time, AIDS Vaccine Shows Some Success in Trials | Hacker News Reader

For First Time, AIDS Vaccine Shows Some Success in Trials

nytimes.com

26 pointstimr14 comments

Some success: 30% effectiveness in that 51 out of 8000 given the vaccine because HIV positive, and 74/8000 given the placebo did. I presume that really is statistically significant despite the chance it could have just been random?

They also don't know how it works...

"Although the difference was small, Dr. Kim said it was statistically significant and meant the vaccine was 31.2 percent effective."

The confidence interval is determined by the number of people in the trial (N=16,400), not the number of people who get sick.

If the rate of infection in the general population was even lower, say 1 in 100,000, and they actually trialled 200,000 people, finding that the vaccine perfectly prevented infection, should they conclude this with high confidence? Or have I misunderstood you?

Besides, Dr Kim is quoted elsewhere as saying "the result could be due to chance" and the vaguely phrased "31.2 percent effective" is not encouraging. Add in the likely funding source of AIDS vaccine trials, the pressure to get results and the unknown mechanism and it doesn't look so great.

The result can always be due to chance. Confidence intervals let you estimate how likely it is that your results are due to chance. In this case, the results are unlikely to be random -- but it's always possible. That's all he was saying. Maybe this study was the 1 in 10,000 that would randomly show a 30% difference in infection rates. Is that likely? No. Is it possible? Sure. Anything's possible.

Confidence intervals get smaller as you include more people in an experiment, but they get smaller more slowly as more people are added. So it's fairly easy to go from a 2% to a 1% confidence interval by adding a couple thousand people, but it takes a lot more people to go from 1% to 0.5%. And of course, it's asymptotic -- you'll never get to zero.

That said, in this case, the point is academic. This study was large enough that the confidence interval is going to be well below 30%.

You didn't understand the parent.

See my post below.

Also, of course the result could be due to chance. You could flip a fair coin 25 times in a row and come up with heads each time -- it's just pretty damn unlikely. Similarly, I'd bet what he's measured is really this: you have two equally sized groups, one treated, one untreated. In group one you get a rate of x, and in group two, you get a rate of y. How big does |y-x| and the size of the two groups have to be to conclude it was not just chance? As below, a t-test can help you answer that.

The key is not to find "who gets HIV" vs. "who does not get HIV." Instead, we are trying to measure "who gets exposed" vs. "who gets HIV." Although we do not know how many people are exposed in each group, we can make sure that the number of exposed people in each group is approximately the same by increasing the number of people vaccinated. If we know that the same number of people are exposed (which is less than the sample size), then the number of people who get sick is much more significant.

By increasing the number of vaccinated people, the number of exposed people becomes more equal between the two groups since a few "more exposed/risky people were assigned to group A" will have a less significant impact.

I have to agree that the numbers don't seem to differ significantly enough to draw any conclusive evidence. I agree with your math in saying that if you had the vaccine, your chance of infection is .6375% and without the chance of infection is .925%. From this perspective I don't see any hard numbers to back the claim. I'll wait for a more better controlled experiment before I believe this one.

I presume that the investigators used something like the t-test for a difference of means. You treat 1/2 the group and don't treat the other; if you selected these two groups randomly, then they should a priori have equal likelihood of infection. You then can create a confidence interval for the hypothesis: are the two means / rates different.

See the second use here: http://en.wikipedia.org/wiki/Student%27s_t-test

http://healthpolicy.stanford.edu/news/even_modestly_effectiv...

link from the AP feed http://hosted.ap.org/dynamic/stories/M/MED_AIDS_VACCINE?SITE...

So the US army is using Thai people as guinea pigs?

They probably sought them out for their high incidence of HIV -- it would obviously be unethical to specifically infect people to test the vaccine.

To add to this, at the time the trial started, Thailand was one of areas with the highest infection rate. It's common practice to do a trial where the disease is the worst.

The article reports that the Thai government (the Thai Ministry of Health) has cooperated willingly with this study. Thailand was an early country in Asia to have a high AIDS infection rate, and has been a leader in that part of the world in working on strategies to find community approaches to preventing the spread of AIDS and to find medical preventives and cures for AIDS.

Reply on news.ycombinator.com