60% of OpenAI model's responses contain plagiarism | Hacker News Reader

60% of OpenAI model's responses contain plagiarism

axios.com

32 pointsmdhb27 comments

I couldn't find the actual methodology.

The copyleaks.com press statement itself is also very sparse on what was actually measured.

https://copyleaks.com/about-us/media/copyleaks-research-find...

Maybe I'm looking in the wrong place?

Though even if we do get their report, looking at the statement and the site in general, it seems these folks are selling a product here.

"A new report from plagiarism detector Copyleaks"... so a plagiarism detection firm says plagiarism is rampant. This is not news, it's a press release reformatted and branded to look like news.

Most things have already been said, so I would think 60% of sentences probably contain plagiarism. “How’s it going”, “weather’s looking good today”, and countless other phrases aren’t original. Plus the longer a response, the more likely it’s got some technical plagiarism in it.

Also, 100% of computer science answers being plagiarised? How is that possible?

It's called "language model" for a reason, the AI is modeled after existing human language corpus.

Maybe there's "plagiarism" for today's version, but given time, the AI could learn from real world itself with multimodal capabilities and eclipse human knowledge in some point.

But is it copyright infringement? This isn't academia, so that's the only test that matters.

> OpenAI spokesperson [...] "We have measures in place to limit inadvertent memorization, and our terms of use prohibit the intentional use of our models to regurgitate content."

If you demonstrate a model obviously violating copyright, not only does that sound like something you did (not the model), but the demonstration also violates the ToS?

What percentage of Google search results contain plagiarism?

None considering it inherently cites the source.

Obtaining citations is necessary no matter what tool you use in an academic setting though.

It turns out you can make it all the way to become president of Harvard [1] while ignoring this rule so it is questionable whether it is as set in stone as you make it out to be, at least in certain disciplines.

In a way these models are a perfect mirror of the current academic climate. They plagiarise without remorse, they follow the latest identity-politics diktat to a point and make up 'facts' when needed to reach a desired narrative. Google Gemini is the latest example [2] of where this leads.

Given that it is plausible that models like these will soon be used in educational settings this is a recipe for disaster. The same goes for the trend to replace search engine results with 'interpreted' results in which LLMs take up the same role as Winston in 1984: Winston works in the Ministry of Truth where he alters historical records to fit the needs of the Party.

It is time for a decentralised distributed search engine which limits itself to pure search, something like YaCy [3]. Something to replace Winstonian search engines like Google and Bing (et al.).

[1] https://www.campusreform.org/article/claudine-gay-is-a-dei-h...

[2] https://news.ycombinator.com/item?id=39465255

[3] https://yacy.net/

Instead of pressing that down-vote arrow in yet another attempt to grey out a dissenting opinion it would be interesting to hear what you have to say. If you do not agree with any of the facts I stated - plagiarism being rampant in academia, LLMs doing the same, identity politics ruling the roost in places like OpenAI and Google - where the lead of the Gemini project is a political activist against 'white privilege' [1] - and the near-certainty that LLMs will soon play a large role in education as well as 'content discovery' - just state what you consider to be the truth.

[1] Speaking in a keynote address in 2021 she openly suggests that she treats “Black, Hispanic and Latinx” employees differently from white employees, she insists Google’s commitment to “antiracism” is a key component of their AI work from which they won’t be deterred, she was caught on hidden camera implying that 'Google will do what it can to prevent Trump from winning again', etc. Treating people differently based on race is called 'racism' and as such it is punishable under criminal law.

> Copyleaks is an AI-based text analysis company that began selling plagiarism-detection tools to businesses and schools long before ChatGPT's arrival.

This is an advertorial from yet another garbage company that just says all text is plagiarized.

Indeed all text exists if you make the samples small enough. Any sentence you can find somewhere, two sentences less, etc.

Their detection system is AI, so it might be hallucinating as well?

These systems are ruining students, delaying their academic journey or straight up causing losses of scholarships, have very high sensitivity because they sell on the promise of detection, and are blindly trusted by design because they go by feeling instead of providing sources for their claims (there are better ones that do, but are often misused as end result providers instead of as a basis to start a research)

Funny how on the other side of the tools you find professor use ai while ostracizing ai, and refuse to define a due process for these tools usage while claiming that student are lacking integrity.

I asked kids who attend my former school a pretty highly reputed one about what the policy for AI stuff was and they said that kids are being expelled for using ChatGPT and the criteria are these AI fake detectors. I would sue the school.

https://www.nextbigfuture.com/2023/08/chatgpt-is-a-glorified...

Is GPT a tape recorder?

Even if you view it that way, it is a very useful tape recorder.

Coffezilla went on a very amusing rant the other day, it would take me quite a while to find but he was reminiscing about his Lex Friendman podcast and I guess Lex kept asking him "what do you think about X" questions that were kind of related to what he knew, but he's not at all an expert in, and he had to keep reminding himself not to answer them. That reminds me of this AI article. Certainly, Michio Kaku is brilliant. At theoretical physics.

>Our models were designed and trained to learn concepts in order to help them solve new problems

Isn’t that claim complete and utter nonsense? It is a large language model after all, not a large concept model.

I've been playing a Bingo game for news articles that aren't negative about AI.

I don't have any squares yet.

Please send help.

I mean, sure. I use it as an alternative to, or alongside, a Google search.

A.I will be the end of us. And what is the application use? To piss people off who want to talk to fellow humans? No thanks. This is just another CEO “gotta have it” it was in forbes! Executives have no place in security or making these careless choices.

I honestly don't understand this downer attitude.

AI is going to make it possible for anyone to make games, movies, programs. It'll automate all the mundane stuff we don't like to do, and it'll let any one of us reach higher than we ever could alone before.

I'm done with churning butter. I'm ready for the flying cars and jet packs.

It’s already possible for anyone to make games, movies, programs.

The problem fundamentally is attention is limited thus all creative endeavors end up having a power law distribution. AI won’t solve that.

It was possible to write before the printing press, too.

The problem is we will probably still be churning butter for even less money. Automation was supposed to reduce the work week to 10-15 hours and it resulted in both husband and wife having to work 40 hour weeks to get by as we created new bullshit jobs to fill the empty space. The cream and time savings went to the top. We don’t want to see that happen again with AI.

AI will be better at writing code, composing music, solving geometry problems, and innovating. Humans will be better at sweeping floors, clearing sewage pipes, and other manual labor.

Reply on news.ycombinator.com