> I don't trust any area of research where the odds ratios are below 3.
Then what’s the point of researching even slightly more subtle effects? This person might as well start smoking: they wouldn’t trust any amount of evidence that smoking causes cardiovascular disease — the odds ratio seems to be 2.
> That doesn't mean I only believe in 3–1 or greater effects. If you can show any 3–1 effect, then I'm prepared to consider lower odds ratios. If teenage girls with heavy social media use are three times as likely to be in the experimental group for depression 12 months later than otherwise similar teenage girls that don't use social media, then I'm prepared to look at evidence that light social media use has a 1.2 odds ratio, or that the odds ratio for suicide attempts is 1.4. But without a 3–1 odds ratio as a foundation, it's my experience that looking at any random data can produce plenty of lesser odds ratios, which seldom stand up.
I found this a little confusing, but I think what Brown is saying is that if all of the effects found in an area of research (not a single study) are small, it's probably not a real phenomenon.
Even with this explanation, Brown is effectively saying that any effect with an odds ratio of less than 3 cannot be studied. I don't think that position makes any sense.
And also, an odds ratio of 1.4 means social media makes it 40% more likely one will be depressed. That is a massive effect. I don't understand why people are saying these effects are small.
If, for example, research shows that social media use increases one's risk of depression by, say, between 32% and 34%, with p < 0.000001, they're going to discount a result that only has a 1-in-a-million chance of being a fluke simply because it can only show a 1/3 rise in risk, rather than a 3x rise?
That's not skepticism. That's dogma.
https://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy
You need a specific hypothesis to test before you start examining (or better, collecting) your data for a p-value to make any sense.
P-value and effect sizes by themselves are not meaningful enough, you should always consider the full context of the experiment, subject of study and domain.
Edit/PS: 3x or more is excessive though.