Hacker News Reader

Comment by tynpeddler | Hacker News Reader

One thing to keep in mind when reading through this, it's a pre-print (not peer reviewed) that's citing other pre-prints like https://www.medrxiv.org/content/10.1101/2021.07.29.21261317v.... It's preliminary work and may change before final publication.

View on HN

i wish there was some way to amplify or emphasize what you have said. this is the pitfall with preprints, then there is the pitfall of preprint supported by citation of preprint.

agreed we should warp speed, but not so warp as to pile hypothesis upon hypothesis.

I wish there was a way for me to transform this statement. The peer review process is a joke. Preprints are every bit as good as peer-reviewed publications, for now.

This will change if pre-prints ever offer bona fide academic points, but for now, there is no difference in quality.

peer review or not, when reading you have to:

* determine whether the numbers are intentionally fudged.

* read the methods and results and check for possible systematic errors.

* see if the data supports the conclusions of the authors.

* make sure you understand the limits and implications of what the authors claim.

as a layperson, I can't really do this for biology. so inevitably, I give things a quick once-over, come up with questions and look to the people I trust for answers or contradictory data. so it's basically an ad-hoc peer review.

That’s like saying QA testing is a joke because it doesn’t catch everything so why not skip it. Peer review catches a lot of issues early but it’s based around the assumption papers are submitted in good faith. Compared to most things you read in say magazines they might be good enough, but it’s really not the same.

Have you seen peer review catch issues in papers you've submitted?

Of the papers I've submitted, exactly one received a quality peer review. The rest were nonsense. Both accepted and rejected papers clearly weren't read. All incentives align to giving a paper a quick skim and being done with it. NIPS showed the process is indistinguishable from noise in an experiment a few years back.

It's like QA testing with a six-sided die.

QA testing would be replication.

Peer review is more like code reviews... but the code is hidden and you can only see the comments.

the whole purpose of a preprint is to confer with others having experience or specialty regarding the subject of thesis. Noone knows it all, or has a perfect version of the contemporary facts. peer review is quality control

hydroxycholoroquine and invermectin as treatment for Covid-19, as well as a vaccine->autism link, are all well supported by preprints. They have all since been disproven, but pre-prints aren't just as good.

Slightly related : how do you explain the (obviously) fraudulent lancet paper claiming for hcq toxicity, and retracted one week later ? (but that was enough time to have hcq be banned in a few countries, including france).

Since this story ( which smells a lot like corruption and lobbyist work) i must say i'm a bit confused about the true value of publication for things related to covid

Editor of BMJ estimates about 20% of scientific publications are fraudulent.

https://blogs.bmj.com/bmj/2021/07/05/time-to-assume-that-hea...

Fewer than half replicate. I would estimate a lot of that comes from softer research malpractice.

Being confused about the value is a fair reaction. It is high; without scientific research, though, we wouldn't have vaccines or monoclonal antibodies. On the other hand, it is lower than assumed, and trusting individual papers and peer review processes is a mistake.

How much value would Facebook have if it were your only source of information? A lot, actually. Imperfect information is valuable too.

Yet this article was published in the middle of the covid crisis on of the hot topics.

I can't imagine the journal's reviewers being careless.

Have you published much?

I haven't seen many careful reviewers. It's an anonymous process, and no reason to put in any effort at all. On the procrastination list, peer reviews are the very last thing scientists do.

I like to say that peer review is what happens after a paper is published.

no peer review is what happens after a manuscript is submitted, after a preprint is circulated, after a paper is submitted to a journal and after publication.

it is an ongoing continuum of process

meh, if you're going to publish something then I'd rather just have people get their papers hosted on arxiv asap and then get feedback from everyone. I have not seen significant value added by journals and the review process, in my field.

When I saw this was the first comment I knew the study showed natural immunity to be better than vaccine induced immunity without even clicking :P

Yes good point, this is early work but definitely a necessary step toward answering an important question.

It's important to note the tremendous amount of evidence from a wide body of peer-reviewed literature which supports the fact that immunity acquired through natural infection provides protection against reinfection that is at least equally effective as vaccination. This fact has been repeatedly demonstrated in multiple large scale and long term serological studies [1][2][3][4][5].

- A previous history of SARS-CoV-2 infection was associated with an 84% lower risk of infection, with median protective effect observed 7 months following primary infection. This time period is the minimum probable effect because seroconversions were not included. This study shows that previous infection with SARS-CoV-2 induces effective immunity to future infections in most individuals. [1] (N=25,661)

- The study results suggest that reinfections are rare events and patients who have recovered from COVID-19 have a lower risk of reinfection. Natural immunity to SARS-CoV-2 appears to confer a protective effect for at least a year, which is similar to the protection reported in recent vaccine studies. [3] (N=15,075)

- Reinfection is rare in the young and international population of Qatar. Natural infection appears to elicit strong protection against reinfection with an efficacy ~95% for at least seven months. [4] (N=192,967)

- The degree of protection (10-fold) associated with seropositivity appears to be comparable to that observed in the initial reports of the efficacy of mRNA vaccines in large clinical trials. [5] (N=3,257,478)

The OP is one of the first and largest scale studies to evaluate the risk of reinfection - in the context of the delta variant which is currently dominant in many countries around the world - by directly comparing naturally acquired immunity to vaccine-induced immunity.

> those vaccinated are still at a 5.96-fold increased risk for breakthrough infection and at a 7.13-fold increased risk for symptomatic disease compared to those previously infected.

> SARS-CoV-2-naïve vaccinees were also at a greater risk for COVID-19-related-hospitalization compared to those who were previously infected.

If the OP study holds up to peer-review (which it likely will), then we can expect further studies attempting to replicate this finding, and even more studies attempting to explain why naturally acquired immunity induces a more robust immune response. The findings from those studies will be leveraged in the design of future vaccines - so these results will be a win just about any way you look at them.

[1] SARS-CoV-2 infection rates of antibody-positive compared with antibody-negative health-care workers in England: a large, multicentre, prospective cohort study (SIREN) https://pubmed.ncbi.nlm.nih.gov/33844963/

[2] Risk of Reinfection After Seroconversion to Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2): A Population-based Propensity-score Matched Cohort Study https://academic.oup.com/cid/advance-article/doi/10.1093/cid...

[3] Assessment of SARS-CoV-2 Reinfection 1 Year After Primary Infection in a Population in Lombardy, Italy https://jamanetwork.com/journals/jamainternalmedicine/fullar...

[4] SARS-CoV-2 antibody-positivity protects against reinfection for at least seven months with 95% efficacy https://www.sciencedirect.com/science/article/pii/S258953702...

[5] Association of SARS-CoV-2 Seropositive Antibody Test With Risk of Future Infection https://jamanetwork.com/journals/jamainternalmedicine/fullar...

I saw some replies to the OP where people were implying that the study results have been known for some time and there's some kind of fanatical, pro-vaxx conspiracy which was suppressing this information. This is what motivated my original post.

Other than that, I agree with pretty much everything you just said. One thing to keep in mind when it comes to policy however, is data quality. The vaccine trials were double blind, controlled studies, which are the highest quality of medical study. This is why the short term, vaccine efficacy results were quickly acted on. By necessity, the OP and the studies you linked are retrospective, observational studies which are considerably more difficult to analyze correctly, and usually provide lower confidence results. Which is why the Israel study is so important (they seem to have high data quality) and why we're waiting for so many of these studies to come in before changing public health recommendation.

One concern I do have about the Israeli data is that the paper on waning vaccine efficacy is using de-anonymized data, which means that for privacy and national security reasons, it's more difficult for the scientific community to double check their results.

Yeah good points all around. Just wanted to mention that even in the high quality phase 3 vaccine trials they found evidence that immunity acquired from previous infection was effective at preventing subsequent symptomatic illness. Obviously that wasn't the primary goal so it was hardly even mentioned in the published papers, but you can see it in the raw data. For example, from the Moderna phase 3 RCT [1] see table S17 on page 42 [2].

Out of the people with natural immunity from previous infection: only 1 out of 337 unvaccinated people were reinfected, and 0 out of 343 vaccinated people were reinfected.

Admittedly they didn't have a large number of participants in this cohort, so their statistical power was limited in this respect, perhaps contributing to their rational for not mentioning it as a primary finding.

[1] Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine (Moderna) https://www.nejm.org/doi/full/10.1056/nejmoa2035389

[2] Table S17. Vaccine Efficacy by SARS-COV-2 Status, Modified Intent-to-Treat https://www.nejm.org/doi/suppl/10.1056/NEJMoa2035389/suppl_f...

Another thing to keep in mind is that a lot of bad science manages to get published even in the best journals. Science always is about preliminary information about the best current understanding of things.

Reply on news.ycombinator.com