A Case of Plagarism in Machine Learning Research | Hacker News Reader

A Case of Plagarism in Machine Learning Research

nicholas.carlini.com

75 pointsEvgeniyZh31 comments

Am I missing something or is this nonsense? The "Big Model" paper is a survey paper that summarizes the current state of the field, while the "original" paper is specific technique. The former, as a small part of the survey, describes what was presented in the original and uses the same language the authors use. Why wouldn't they? Why would they need to re-phrase the facts and methods the authors described themselves when all they're doing is giving an overview?

They are not claiming to have done something they didn't.

Same argument about copying super generic introductions like "Deep Learning has been successful at...." - does each author really need to come up with their own variation of widely accepted facts to avoid "plagiarism" ?

You should describe the same concepts, but you should use your own voice. A survey paper is not just copy pasting blocks of text and adding connecting language. A survey itself should have its own story and therefore your own voice. You're allowed to quote, sure, but what is shown is beyond a reasonable quotation and the "quoter" doesn't reference that the section was quoted.

Yes, you are missing something. No, this is not nonsense. It’s considered academic plagiarism to copy paste text without indicating the passage is a direct quote. They should re-phrase the text. Not only is it academic plagiarism, it’s also poor writing, because the copied text is providing a different framing than the overview article, and because the copied text is in some places incoherent due to formatting (e.g. we show that over 1 [sic] of the data most models emit).

You are wrong, the Big Models paper does in fact claim to have done something that they did not, e.g. “We introduce two complementary methods for performing deduplication.” They do not introduce these methods. The text they are lifting did.

What you are missing is long standing norms around academic plagiarism and false claims in the Big Models paper (as a result of copy pasting language).

To quote the article:

> But even putting aside the fact that claiming someone else's writing as one's own is wrong, the value in survey papers is in how they re-frame the field. A survey paper that just copies directly from the prior paper hasn't contributed anything new to the field that couldn't be obtained from a list of references.

Good survey papers can be important contributions in their own right (e.g. [1]). A good survey should contextualize works within a subject area with respect to each other and identify high level trends/ideas in that subject. These connections are not only useful for learning a topic, but also for positioning novel work or identifying under-researched areas to focus on.

If the authors felt that one of the papers they plagiarized concisely expressed what they wanted to say, they could simply quote and cite that work. Otherwise, it could be construed that the authors are claiming to be the ones drawing the conclusions they wrote. Moreover, from the article, the survey in question seems to be pretty egregiously plagiarizing, which deserves to be called out/shamed.

[1] https://arxiv.org/abs/2111.11426

I disagree with this:

Whether or not a survey paper is "good" is irrelevant here. Yes, a survey paper that just lists others papers may be a bad survey paper, but it does nothing wrong as long as it cites the original papers, which this does. A bad survey paper may not be published in a journal, that's what peer review is for, but there is nothing wrong with publishing it openly on the web.

And there is still value in aggregating other papers, even if it's just a list with description. That's the reason why these "awesome-XX" Github repos are so popular. Time to hunt them down?

If you look at the plagiarized language in the article, it seems as if the BM paper authors are claiming contributions (emphasis mine). Credit is a major currency in research, and it's important to give it where it is due. If someone did this with one of my papers, I'd be quite upset.

For example (Emphasis mine):

> The risks of data memorization, for example, the ability to extract sensitive data such as valid phone numbers and IRC usernames, are highlighted by Carlini et al. [41]. While their paper identifies 604 samples that GPT-2 emitted from its training set, we show that over 1 of the data most models emit is memorized training data. In computer vision, memorization of training data has been studied from various angles for both discriminative and generative models Deduplicating training data does not hurt perplexity: models trained on deduplicated datasets have no worse perplexity compared to baseline models trained on the original datasets. In some cases, deduplication reduces perplexity by up to 10%. Further, because recent LMs are typically limited to training for just a few epochs

Yes, I agree that's bad but looks like sloppy copy and pasting as opposed to intentional plagiarism to claim contributions. Would it have been okay if they said "they" instead of "we"?

Then who is "they" in this situation? You need a citation!

>>summarizes the current state of the field, while the "original" paper is specific technique.

They copied the summarizes from that those techniques.

>>Why would they need to re-phrase the facts and methods the authors described themselves when all they're doing is giving an overview?

A review paper is synthesis of ideas and they copied the synthesis of other authors.

Ultimately, this is what plagiarism looks like in the case of a review paper.

This clearly falls into the federal government's definition of plagiarism.

(I'd know, I just had to do a mandatory course for grad school on research ethics, exactly this scenario was used there as an example.)

>A survey paper that just copies directly from the prior paper hasn't contributed anything new to the field that couldn't be obtained from a list of references.

IMO the whole point of a survey paper is to avoid having to read through a whole list of references. There is a benefit in extracting information from those papers and assembling them together into one thing. It's expected that the paper is going to be a compilation of information from the papers that it is surveying.

This seems to be a case of a bad and a lazy survey paper rather than plagiarism. Plagiarism implies misrepresentation of some kind -- usually passing someone else's work as one's own. With the original works cited this does not seem to be the case here.

There are more egregious dirty laundry that perhaps needs more attention than this particular one.

While I was a grad student, a labmate's research paper was copied word for word, at least the abstract, introduction and conclusion. If memory serves, this was in around 2002. Funnily enough this plagiarized paper submitted to a conference happened to get assigned to my same labmate for review.

The are chairs were informed and the paper rejected, but more interesting was the discussion that ensued within our own lab with other Chinese labmate's. They mentioned that in Chinese universities, at least then, it would be considered perfectly kosher piece of work by a student to replicate word for word a good paper that the student had expended effort in identifying, such an act would not be considered plagiarism, perhaps even a demonstration of good work ethic. There was no clear consensus on whether submitting it in their own name in a prominent ML / datamining conference would have been perceived as an act of plagiarism.

My labmate who's paper got plagiarized got lucky that it came to him for review. I can easily imagine scenarios where it needn't have. Also its only because it was submitted to an international conference that it got caught out. I wouldnt be surprised if more of these were going on in more home brew conferences.

The original paper was published in ICML. I dont seem to recall with high confidence the exact conference the plagiarized copy was sent to.

I thought journals use some software to detect plagiarism by scanning the submitted files and checking for copied text. Of course, it's not as straightforward because of all the symbols and math involved. But I think whether the files are submitted as Latex or Word docs, Regex must help extract pieces of text and reverse search them in previous papers. The problem is that this necessitates scanning previous papers too, which journals might not be able or willing to do.

Anyway, I always thought research in ML is suspiciously fast paced, but most published work is honestly bs. This post says one reason for this might be the fact that most research isn't actually contributing much to the literature, but rather reiterating previous papers.

This paper doesn't seem to have been submitted to a journal/conference and probably didn't go through a standard peer review process.

As bad as this is, someone lying about their results in a paper is FAR worse than them copying and pasting was is essentially boilerplate as far as the authors and reviewers are concerned...

And we all know that outright lies are all too common in ML research...

Also a copyright violation. ArXiv should remove the article.

The individuals listed as authors of the paper who blatantly plagiarized the original research happen to work at prominent government organizations, educational institutions, and major corporations:

* Beijing Academy of Artificial Intelligence

* Tsinghua University

* Wechat, Tencent Inc.

* Northeastern University

* Renmin University of China

* Peking University

* Huawei TCS Lab

* Institute of Computing Technology, Chinese Academy of Sciences

* Shanghai Jiao Tong University

* JD AI Research

* Harbin Institute of Technology

* Columbia University

* ByteDance AI Lab

* Microsoft Research Asia

* Mila-Quebec AI Institute & University of Montreal

* New York University

* BeiHang University

* Institute of Software, Chinese Academy of Sciences

* Institute of Automation Chinese Academy of Sciences

IMHO, the main authors, who copied and pasted text without giving credit, should be fired on the spot.

Which of the 100 authors did the copying?

I find it hard to believe that all of them collaborated on the paper or even aware that their name is on it.

Going to be hard to say. The paper itself is 200 pages. I also wouldn't be surprised if more papers were copied in a similar manner given the size of the manuscript. If the authors are using some version controlling (insane not to with this large of text) then it should be relatively easy to check who did it and then check the rest of their writings. But that has to come from internal pressure.

I am a senior researcher in the same area. From the perspective of ethics enforcement on an institutional, state, or field level, it does not matter which author it was that committed the act. As a co-author you “sign off” on the paper as a whole and is expected to do due diligence to ensure that the paper in its entirety is up to the standards we expect. This is research ethics 101.

Now, in practice I would expect that you get off easier on a paper with a ridiculous number of authors such as this one as long as you find the perpetrator or a fall guy. Lastly, there is also the issue of papers at time being submitted without the consent of a co-author (possibly without them even knowing of the paper), which could complicate the matter further.

Shame on whoever did this, among the authors from prestigious institutions such as these there ought to be at least one to stop this kind of stupidity or at the very least be competent enough to write the very same text without copying.

It is probably a sloppy mistake, ineptness, and poor process controls by a contributing team, not pure malevolence. In a survey paper some people are sloppy and copy/paste source material directly into their file and edit it, or instruct others to do edits with notes. This creates a large risk that the edit never happens or the wrong version is submitted.

Edit: also take into account plenty of these researchers write English as second or third language. Copied text won't necessarily sound as obviously out of place and process becomes even more important.

To be frank, it does not really matter if it is pure malevolence or not. I am to be honest a bit more stringent than many of my colleagues, but even if I do not hold everyone to my standards I expect my fellow scientists to have high enough standards so that I can read (and most importantly trust) their writing and findings with confidence. Yes, the pressure to publish is insane and even more so in some countries than others (curse those in positions of power that cause this), but as it is impossible to verify every written statement or finding in any science, we must rely highly on honesty and high standards to maintain trust whether we want it or not as there is not (and should not) be a central point of authority to dictate good science. If we fail to do so – regardless whether by malevolence or not – we will be lost as our mode of consensus and communication will sooner or later be in tatters. Thus, screw behaviour like this, it should not happen and the senior academics on that paper should call for its retraction until it is in a state where it can be consumed by the community.

I agree retraction or correction is appropriate here. However, I would argue that correcting mistakes is a routine element of science -- whether due to a copy error or a miscalibrated instrument -- whereas malevolence is not.

Sure, the end result on the consumer's trust in the science is the same regardless of why the deficiency occurred, but the problem is far more tractable if it results from sloppy QA (or equivalently calibration error and calculation issues) rather than malevolence. In the first case the researchers can eat some crow and do better science next time; in the second you have people who flat out don't belong in science.

Your stance on ethics is correct. However, in practical terms, not all co-authors of a paper even read the whole paper. Most often, they contribute sections of the paper.

I don't think all co-authors are equally culpable in this. The paper should be retracted, but the punishment should be only to the actual perpetrator.

Frankly, if you do not have it in you to read through a paper you are co-authoring, maybe you should not be a co-author or maybe you should not author massive overview papers? Did someone hold a gun to their head forcing them to author a work that was so large in scope that they could not maintain sound academic standards? If you want the credit that comes with authorship, I expect you to “pay” by maintaining proper standards.

Section 2, which contains the first match, lists only six authors.

> Renmin University of China

This is an interesting choice of English name. You don't often hear about the Renmin Republic of China.

The credits are very likely still in the references section at the end.

As an adult, with a PhD, doing research in machine learning I have difficulty reading a paper that's all about BMs ...

BMs?

Reply on news.ycombinator.com