AI-Generated Garbage Is Beginning to Pollute Scientific Literature

"The researchers hypothesize that GPT-style generative models, and similar types of language generation frameworks, have been used to produce much of the text in the flagged papers; however, the way that a generative model abstracts its sources makes this difficult to prove, and the chief evidence lies in a common-sense evaluation of poor and unnecessary synonyms, and a meticulous examination of the submission’s logical coherence."

This sounds extremely unlikely.

First, scientific fraudsters are typically extremely lazy; they do not use fancy methods like writing generative models of fake data to ensure the numbers, say, add up to 100%, much less neural nets. (Do we believe that someone would go to the effort of setting up GPT to generate fake papers and then be satisfied with results like 'profound neural organization'? Easier to write such garbage by hand!)

Second, the given examples are so bad I'm insulted on behalf of GPT: 'profound neural organization' looks like a word-by-word calque using a thesaurus or particularly antiquated machine translation (in a "the vodka is good, but the meat is rotten" way), no contemporary AI involved; GPT would not generate such garbage.

Third, if GPT was generating these papers, it would be easy to detect by looking for top-k signatures, using GPT-targeted classifiers (they get >90% accuracy), or looking for stylistic tics of GPT; that they don't provide any specific GPT-like aspects of the text suggests there are none.

The examples of "failed attempts at creative synonyms for known phrases in the machine learning sector" seem the output of translating the text multiple times (Language1->Language2->Language1), more than using a language model.

Why would GPT-3 or another language model use "profound neural organization", which is a non-existing term that it has never seen during training?

This is a really interesting article, but I can’t ignore my pet peeve:

It has a section titled “Not Just Semantics,” but of course they really mean “syntax.” Semantics is meaning!

I absolutely love the way GPT-3 re-words common phrases. Examples from the article:

artificial intelligence (AI): ‘(counterfeit | human-made) consciousness’

ant colony: ‘subterranean insect (state | province | area | region | settlement)’

It's so clinical and literal. It reminds me a lot of the way Overwatch phrases things in Half-Life 2. For example, when you use a swarm of antlions (giant alien insects) to stage a prison break, Overwatch says

"Overwatch acknowledges critical exogen breach. Airwatch augmentation force dispatched and inbound. Hold for reinforcement"

"airwatch augmentation" sounds much more ominous than "air support".

I would be interested to see what GPT-3 could come up with if explicity tasked to do this type of re-phrasing. What happens if you ask it to pretend to be an alien intelligence?

This is as pointed out translation spam.

Which uses whatever AI the big translation services use.

What's surprising is this is many years old on the web but I'm not sure it's been written up?

Massive 'news' blogs use it. Most stories will get a translation spam version.

Basically it allows copying without obvious copyright issues and might get great SEO. Not sure how on top of it Google is, some articles are saying Google can't detect it.

Even if Google is on top, Bing mightn't be able to find Google translator spam without Googles help/breaking Google's TOS

It could be worse. What if our AIs generated high quality papers?

we could just move that information onto a site where scientists have to verify their identity, we could also build a science blockchain and verify identities that way as well