Just like with the Netflix Prize stuff, where the conclusion was very similar, ie. just dump in as much data as you can, crank up the ML machinery, and it'll discover the features (better than you can engineer them) and learn what to use for recommendation ranking. And that's basically what we see with GPT-3 too. If you have some useful labels in the data it'll learn them even without supervision, because it has so many parameters, it basically sticks.
Get some papers run it through a supervised training phase where you give it a set with every paper scored based on how retracted/bad/unreplicating it is and you'll get a great predictor. Then run it to find papers that stick out, and then have a human look at them, and try to replicate some of them to fine-tune the predictor. Plus continue to feed it with new replication results.
That said, using an ML system as the gatekeeper as OP suggested is a bad idea, as it'll quickly result in the loss of proxy variables' predictive power.
Though ultimately a GPT-like system has the capacity to encode "common sense".
If you look at what GPT produces closely, a lot of it is simply bullshit. It sounds plausible but is wrong. That's exactly the wrong type of AI to detect plausible-but-wrong-bullshit papers, which are the most common type.
The language model (GPT-3) doesn't have to understand physics, it just have to help extract out some semantics of the paper.
There needs to be a classifier on top trained with a labeled set of good and bad papers.
https://arr.am/2020/07/25/gpt-3-uncertainty-prompts/
It's really cool how the uncertainty prompts alter the confidence associated with the next words.
I guess I'm not disagreeing with you in the abstract that a theoretically strong enough AI could identify bad papers, especially if it had some help for 'real' arithmetic. It at least could flag the most basic issues like plagiarism, cited documents that don't contain the cited fact, etc. Detecting claims that are themselves implausible seems like the hardest task possible, however. That's very close to general AI.
Yes, of course. I was simply trying to say that an AI can be quite successful in detecting the usual no-nos, eg. multiple comparisons without correcting for it, p-hacking, or ... who knows what "feature" the classifier would find. Maybe there's simply none, so it'll be really up to subject matter experts to review them. (But it's unlikely, because there are quite successful blogs devoted to simply picking apart shoddy papers simply based on looking at the controls, and other parts of experiment design and the methods sections, and of course the aforementioned stats.)