Comment by taleinat | Hacker News Reader

Comment by taleinat | Hacker News Reader

I'm very skeptical of these results: The sample size is much too small!

They started with only 64 individuals. Of those only ONE was found to be especially attractive to mosquitoes, and only TWO especially unattractive. They then used the scents from these three individuals, and 5 others who did not participate in the initial testing, for the following phases of research.

Also, of the 5 additional people, two were found to be especially attractive, and one unattractive. This is extremely inconsistent with the results from the first phase.

Now, they did a ton of work and show very interesting results. But these problems make all of their results very questionable in my mind.

Yes, followup research could shine more light on this to validate their results (or not), but I wish they would make less grand declarations about the meaningfulness of their results. I bet this is going to make the news around the world... and if it later turns out to be inaccurate, it's only going to further reduce the public's trust in science.

I don't understand what exactly you are complaining about. Can you elaborate on which specific statistical claim you think they should not make?

This is not an analysis of "what fraction of people are attractive to mosquitoes", it is about how mosquito attraction differs between people. You can make meaningful statistical claims about this with just two subjects (and lots of measurements, which they did - see e.g. Fig 1G).

I also don't understand how you complain about the insufficient sample size and then go on to claim the difference between two cohorts is "extremely inconsistent".

> You can make meaningful statistical claims about this with just two subjects

With just two subjects, there is a very high chance that the results of your testing are overly specific to those two subjects, and do not hold for most of the population.

With only 8 subjects, as in this study, that is still very true.

For example, people like the one person the mosquitos were extremely attracted to could be one in 10, one in 100 or one in a million. In the latter case, the findings are much, much less meaningful.

That's true even for the findings about specific genes affecting attraction. What if that one person is an extreme outlier and the mechanism causing the results isn't relevant for 99.9999% of the human population?

> Can you elaborate on which specific statistical claim you think they should not make?

For example, the claim that "Highly attractive people have higher levels of carboxylic acids on their skin" does not seem to be well enough backed by evidence in this case. If they wrote, "the one/three attractive people we tested had higher levels of carboxylic acids on their skin", I'd have no complaints.

> For example, the claim that "Highly attractive people have higher levels of carboxylic acids on their skin" does not seem to be well enough backed by evidence in this case.

I might be missing something, but do you not see additional evidence for this claim in the discussion around figure S4?

In my reading, the authors do go out of their way to point out that carboxylic acid presence is only one of possibly many more factors. (See also "Limitations of the study"). Most of that nuance is lost in the "highlights" section though, I would agree.

(I'm not GP) My answer would be that it's the difference between "Why do mosquitos have differential attractions to humans" vs "Why do mosquitos like Subject 33 more than Subject 28." The untestable assumption is that what makes Subject 33 more attractive to mosquitos than Subject 28 is generally applicable to the population at large. I agree intuitively with GP that finding 1/64 to be highly attractive in one sample and 2/5 in another would be surprising if this were following some binomial distribution, but the methodology for determining attractiveness was different between those participants (live test vs exposing mosquitos to a nylon that had been worn by the participant).

If the professor from my statistics class is anything to go by, you just need a sample of more than 32 and you can assume it's a normal distribution /s

Statistical literacy is stupidly low. Without that, what is reasonable to expect for scientific literacy?

Reply on news.ycombinator.com