Hacker News Reader

Comment by viraptor | Hacker News Reader

Why do you think adjusting the model for: "Nuisance variables were age, sex, racial-ethnicity, gender, handedness, first language (English vs other), country of residence (UK vs other), education level, vocational status and annual earning." doesn't cover what you ask for here? (including graduate degrees) With the described adjustments and enough data points, the distribution of infection cases shouldn't matter.

Or are you making some other point I'm missing?

View on HN

Statistical adjustments at a population level will be much less useful than the pre post design above.

Getting estimates from the same people will provide a much lower variance estimaion of any potential effect.

Also, adding hospital records to this study would potentially provide a better estimation.

All studies have flaws, but this one does look pretty reasonable (the principal components method used should have been better described, for instance, and confidence bounds on the effect sizes would have been super useful).

The "pre post" is not much of a design, unless we've had a long-term practice to force random healthy people into scanning their brains, or we have a time machine.

There are always medical studies running for other purposes that involve doing brain scans on otherwise healthy people. One of my relatives participated in such a study. Presumably those same subjects could be enrolled in a "pre post" study for COVID-19 effects.

Nonetheless, it would be a better design. Horribly impractical, but really useful.

Correcting a model is always a somewhat subjective process that can introduce bias, especially when it is done after the study is completed pre-publication. I am not at all opposed to such adjustments and I was actually involved in a research project that improved on pure chance based randomized selection of trial participants into different buckets (substance/placebo) by weighing in the necessary adjustments in the selection phase. That is why I’m intimately familiar with the state of the art in academia. Novel approaches, such as the one I worked on, are unpopular with reviewers which makes them unpopular with researchers even though they would allow, for faster trials, with smaller cohorts or alternatively to find smaller effect sizes. My criticism of the OP Study however is not the adjustments per se, which as you point out seem reasonable (although we don’t know what alternative corrections they could have made with the available data and what impact that would have had on their findings)but the fact that they had what was needed to conduct one of the rare properly executed longitudinal studies on this question and wasted that opportunity.

With so many variables to adjust for the error bars should be huge.

The paper discusses how the variables were handled and lists the error for estimates. Is your objection that they calculated it wrong? Or that/how they used a linear model for adjustments? What's the range for "huge"?

I'm saying that there's no reliable data to establish how those variables impacted the final results. The correction factors are mostly just guesswork. Signal to noise ratio is low.

Basically this is junk science. The only proper way to do such a study would be to compare the same subjects before and after infection. With so many cases such a study should be feasible.

We don't know what the SNR is. It would be great if the paper included the linear regression values and fit analysis. It could be rubbish or not - but we don't have the data to confirm or deny that.

I get the objection, but I'm getting annoyed at comments that state what's invalid with the study where we don't have the data available for that claim. What's wrong with "we don't know if the study is valid since we don't have details on X"?

That's not how real science works. The burden is on the paper authors to prove they have adequately controlled. From what I can see they haven't done so.

Overall the majority of COVID-19 clinical research has been rushed and very low quality.

Reply on news.ycombinator.com