Google Scholar search: "certainly, here is" -chatgpt -llm
simonwillison.net
simonwillison.net
Pick one of the Google Scholar results from the article and you'll find something awful like this (the start of the second paper): "The carriage of goods under international commercial law is a complex and essential aspect of international trade."
Thus therefore to provide an example thereof one may replaces all instances of "important" or "essential" with "splendid", "brilliant" "magnificent","gorgeous" or "sophisticated".
—"The carriage of goods under international commercial law is a sophisticated and gorgeous aspect of international trade."
This is very interesting. It could also be an effect of Grammarly, which suggests these replacements for the generic "important." Perhaps it's a combination of several effects.
Looking at the search results, none of these are in "real" science journals. This is just yet another way in which predatory journals are terrible for science and academia (and its public perception), but the existence of garbage journals has been a problem for 2 decades (or more?).
On the bright side, productivity at respected institutions has not been significantly hampered by predatory journals. On the other hand, academia as a whole has obviously failed to teach incoming members that this is basically fraud (partially due to the publish-or-perish culture that is now even endorsed by governments).
That being said, most scientists are very smart and recognize the power of ChatGPT. They are most likely heavily relying on it when producing summaries, abstracts, hypotheses, etc., and because they can and will edit it you won't be able to tell from words alone.
ChatGPT is an equalizer between native English speakers and scientists speaking English as a second language.
But both groups will use it heavily and nobody can foresee what the net effect of it is on science.
Having had that experience, I can tell you that it takes weeks or months to write up results as a paper and go through the most boring steps of formatting and organizing them into antiquated presentation formats: Intro/Discussion/Results, dumbass citations, labeling, backward wordings ... etc
What should be a five-paragraph discovery on a blog has to take the form of a multipage, heavily formatted document that contains enormous, unrelated, and useless information.
With ChatGPT, you can do it in a day. If you are not using ChatGPT, you lose out.
Maybe even one day society will re-value conciseness and precision of thought over linguistic diarrhea.
However, I draw a firm line at using them to generate complete academic works or nonsensical content, as that undermines the integrity of research and renders it devoid of originality. LLMs should serve as invaluable assistants to free up scholars for higher-order analysis, not as replacements for human ingenuity.
It doesn't actually bother me if some of that fluff is ChatGPT-generated, provided the author actually read it and accepts the autogenerated content.
But better still, cut the fluff.
But it has been done for at lest a decade or so I think
The idea behind writing a lot has been that it forces students to think systematically and thereby learn. But they are not typically interested in that, they just need a diploma in order to get a job. Before computers they were copying books verbatim and since plagiarism software they paraphrase books. And now chatgpt removes the middle man - the student. It doesn’t make much sense either way. The source of the problem is really the conflation of higher learning with vocational training.
https://dbrech.irit.fr/pls/apex/f?p=9999:1::::::
It's a constant crawl of the literature that uses hand-written phrase detectors. Many of the phrases it looks for are caused by using spinners on plagiarised text, like this one:
https://link.springer.com/article/10.1007/s11042-024-18524-1
"At first, the input picture is transformed into the ruddy, green, and blue organize, and the clamor within the green band is evacuated using a median channel"
Here "noise" has been turned into "clamor". Nonsense text like this is easy to find in journals published by well known brands, in this case Springer Nature. A search for "profound learning" (deep learning) yields over 2000 results:
https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&as_ylo...
The industry is ripe for public prosecution when an article submitter didn't read it, an article reviewer didn't read it, neither did the publisher yet it is published in a journal with a subscription price in the thousands payed by publicly funded libraries.
That rule no longer works. People are still going to rely on it for a while though, and I'm worried it's going to break some stuff over the next few years.
For example, many academic papers are pages and pages of background, discussion, citations, etc. Very often the actual new material can fit on an index card + 1 figure. I understand why all the background is needed, in the abstract, but maybe this traditional way of thinking about incremental academic thought needs to shift in light of the reality of LLMs: LLMs will be used to write needless fluff. Indeed, it's what they're good at.
I'm a patent attorney and I can say that the same thing applies there. Patents are enormous documents filled with legalese, background, description, vague, conditional language, etc. There are good legal reasons for that, but again, often the actual novel invention can fit on an index card.
For that reason, it's self-evident to me that my job will be replaced, at least to some significant extent, by LLMs. But why not just cut out all the needless words? I could imagine patent law that added a "parsimony" requirement to patent filings. Examiners could reject filings that contained extraneous words. Then maybe patent lawyers would still be needed: to write the shortest possible patent that still contained the novelty.
This is just a toy thought experiment, not a serious proposal. My point is that LLMs expose as trivially bankrupt some aspects of academic and patent writing (and probably other places: marketing copy, some nonfiction essay-style writing, etc.).
Maybe LLMs will force us to embrace or even require brevity and conciseness to ensure that human beings stay in the loop.
These aren't purely bad things! I wonder what future social conventions for this kind of thing will look like.
I'd say by definition the answer is no, since it's no longer humans communicating with each other at all.
It implies that the authors did not care to thoroughly review related work. They basically didn't read any papers.
If you try to have a technical conversation with this sort of a person, they'll communicate their points out of their behind.
But the examples cited in the article are indefensible. The authors of the paper we either so care less they did not notice the the “certainly, here is…” preamble made no sense in a paper. Or they have such a poor grasp of English that they are unable to comprehend what they copy-pasted to notice. Either way this suggests that nothing in the paper can be relied upon.
I'm perfectly fine with this usage, although it is very sloppy if the authors and reviewers didn't pick the "certainly" in the final text.
But I was talking about the intended usage, it doesn't sit as fraudulent to me in the scientific sense.
But a quick browsing shows that not all these papers contain that phrase and not all that do seem to be machine generated.
- "[...] Certainly here is a vast and important project for research."
Versus:
- "Certainly! Here is the text with spaces added after each word:"
Second hit: "Certainly! Here is a review of the literature on the carriage of goods under international commercial law:"
Third hit: "Certainly, here is the tabular representation of intraocular pressure (IOP) at 1 and 4 hours for Group X and Group Y:"
Fourth hit: "Certainly, here is a list of both Geo-natural and geosynthetic materials commonly used in civil engineering and environmental applications."
Fifth hit: "Certainly, here is a concise summary of the provided sections:"
Seventh hit: <not found in paper or abstract>
Eighth hit: "Certainly, here is the essential information provided for reference to the proceedings of the conference on the topic of university book production:"
Ninth hit: "Certainly, here is the methodology for conducting a literature review on the attitudes of healthcare workers toward COVID-19 vaccination:"
Tenth hit: <paid access>
Eleventh hit: "Furthermore, to confirm the advantages of the new model, Certainly! Here is the rephrased sentence with improved language quality and clarity:"
I don't think that all results contain such wording, and surely daghamm's statement can't be disproven by listing these. But there is something that (also) rubs me the wrong way about these sentences getting into research papers.
Re-reading Simon's post, the only word that I think others might find contentous is "huge". Would it be better if it read the search expression turns up "a number" of papers, without a qualifier?
I randomly picked 10 articles and all of them had the phrase in a manner that indicated it was directly pasted from ChatGPT.
https://scholar.google.fr/scholar?hl=fr&as_sdt=0%2C5&as_ylo=...
Leave reviews for quality papers by top experts in the area.
So in both cases, you don't make money directly from publications, but from the prestige they bring and the credibility they're associated with ("this guy must be an expert on X, because they published on the topic").
Understandably, concerns about the quality of LLM-generated content are valid. However, it's crucial to recognize that academic research goes through rigorous processes before publication, including oversight by advisors and peer review. These layers of scrutiny help ensure that any content, whether initially drafted with the aid of AI like GPT-4 or not, meets the high standards of accuracy, reliability, and originality expected in scholarly work. AI tools are used as aids in the research process, not replacements for human intellect, and the responsibility for the final output always rests with the human authors. Hence, the academic community already has strong mechanisms in place to filter out inaccuracies or "bullshit," ensuring that the integrity of published research is maintained.
What do we even do about something like this? Is there an organization that has the power to go through this list and blacklist each of the authors from future publications? Our society should have a zero tolerance policy for drivel like this. It cheapens the entire institution.
Before LLMs, one of the main deterrents of low quality work was that the effort required to write a low quality paper is within an order of magnitude compared to the effort for a high quality paper. Using LLMs, however, I can "write" hundreds of low quality papers (that look okay-ish on the surface) in the time it takes me to write a single high quality paper.
I also don't think that researchers publishing high quality papers will benefit that much from LLMs. I tried getting ChatGPT to reproduce an argument I made in my PhD thesis and it took me many tries before ChatGPT even produced something without factual errors. As for padding: Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
I've been thinking lately that one could probably leverage the same technology to evaluate the quality of research, possibly as a post-hoc verification.
> I also don't think that researchers publishing high quality papers will benefit that much from LLMs.
Oh, but they do! If not for anything, LaTeX formatting is tedious.
> Papers should ideally not contain any padding. If a section in your paper is so devoid of actual information that an LLM can write it, you should probably remove it altogether.
It is mostly correct, except for perhaps the abstract and some parts that provide structure for the paper (Outline, maybe Conclusion, ...). However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement. It can seed writing: there is no obligation to copy-paste and leave it as is.
I doubt it. They'll be just as effective as ChatGPT-detectors are. Completely useless.
> If not for anything, LaTeX formatting is tedious.
LaTeX formatting is a very, very, very small part of publishing research. At least it was for me.
> However, you can always seed an LLM with the bits of information you want explained and use the result as a basis for improvement.
That's what I tried to do with the argument in my PhD thesis. It didn't work. Also, at the point where you have meticulously thought through how to structure your argument (and that's what I needed to do to get ChatGPT to produce anything of value at all) you did 95% of the work already. Actually producing the text isn't the hard part of writing.
My point still stands: LLMs help way more if you want to produce low-quality papers than when you want to produce high-quality papers and as a consequence, the percentage of low-quality work will increase.
I wonder how many people adeptly use GPTs and manage to navigate through their academic landscapes nowadays. I bet its majority
Better queries would include proper timeframing and all LLM-preference-tuning phrases from https://github.com/AlpinDale/gptslop
But even then, I don't think this is the gotcha OP thinks it is: LLM generation and summarizing is a legit tool in the paper writing process, especially for conclusions.
In my academic experience, the paper introduction and conclusion is often written last-minute, close to the submission deadline. While a good intro and conclusion are important for the reader, very little time is made available for it since researchers spend maximal time on expanding material findings and experiments. No wonder that assistive writing tools are used to speed things up. As long as proper post-editing and proof-reading is applied.
The whole point of the original article is that people aren't doing even that.
Plus in an academic context, it's important to acknowledge when parts of the paper are not your own work, especially in cases where the generated summary may not accurately reflect the actual paper. Saying that ChatGPT has been used to generate summaries would allow people to judge if the summary alone can be trusted to represent the content of the paper.
Which already filters after 2023.