Hacker News Reader

Comment by Davidzheng | Hacker News Reader

But Tristan doesn't even want to be credited for the millennium prize--does he? He wanted to be the first to solve it and got scooped (which ig is not a great look for OAI ethically but also not forbidden). And the only reading for her of asking to remove levent must be in light of this offer (to credit him for millennium prize) right? It's not like they can ask him to remove levent from the main papers without this offer(what would Tristan gain?)

My guess is that OAI tried to be "generous" and offered to share credit on millennium with Tristan but not Levent. And Tristan got understandably offended by this offer (which probably oai felt like was the right thing to offer but they couldn't really offer to do the most ethical thing for some reason) and then the random conflicts and weird threats started.

View on HN

my reading was that openai did not deny plagiarising the approach from their prompts.

openai then tried to effectively bribe buckmaster with a shared citation, whilst dropping his co-author who works for anthropic.

after buckmaster refused, openai tried to threaten him.

Oai denies looking at prompts but doesn't deny training on them.

if they do not deny training on them, they can't deny plagiarism.

By that logic everything any LLM spits out is plagiarizing the vast majority of work written prior to a few months ago. That doesn't seem like a useful or desirable line of argument to me.

Just replace the model with a human student.

"Training" on textbooks => fine

"Training" with unpublished notes from another professor, then publishing something on that exact topic with a similar approach without giving any credit => extremely questionable.

Presumably the professor voluntarily provided the notes in this analogy. I think the student would also be expected to cite the textbook if building off of it directly. In contrast, humans are generally not expected to cite "general inspiration" or what have you. So if we're to apply human standards, and assuming that the model was trained on the relevant work, it would only be plagiarism if the model directly built upon that previous work (at least IMO).

The trouble here is that if LLM training constitutes direct use then approximately _everything_ they output is blatant plagiarism, not just a few pieces of academic work.

Conversely if training is viewed as analogous to a student attending classes to learn general concepts (not a perfect analogy, I realize) then nothing they output on their own (as opposed to receiving as part of context) is plagiarism.

Thus this seems like a fairly useless line of argument to me as far as the current topic goes. It either implicates this academic work along with literally everything else or else it does not implicate this academic work. Kind of like nuking an entire city and then saying "mission accomplished, killed the bad guy".

This is just a nonsense line of reasoning. Training based on the solution to the problem (or the key insight behind the problem) is clearly a form of plagiarism.

What about my line of reasoning is nonsense? I made no claim either in support of or contrary to yours. Rather I pointed out that by this logic literally everything that an LLM spits out is plagiarism of the vast majority of the entire body of human literature in existence. Can you offer meaningful refutation of that observation of mine?

What does it matter? We're supposed to not call it plagiarism anymore because it's inconvenient to call it the plagiarism machine? What's your actual argument? Otherwise it's completely irrelevant what an LLM does in other contexts or what we call it

Many do indeed hold the position that all LLM output is uncopyrightable plagiarism. They're probably right, but there's an even stronger argument here:

Science papers of a phd level must contain:

1. one or more novel insights

2. a long list of citations to contextualize them and

3. some work to prove that the insights are in fact meaningful

---

In this context, consider a prompt based diffusion model which, when asked, will happily produce a few pictures of a horse in orbit. You then tell it "silly robot, horses can't breathe in space" to which it adds the necessary space suit in a follow up image.

That image is twice plagiarized:

1. the model did not come up with the original idea of putting a horse in space, nor with insight that horses need a space suit

2. the model failed to cite where it pulled the "horse" and "space" concepts from.

It merely did the work (3) to combine the concepts using the user provided insight.

---

The implied accusation here is that OpenAI used the insights from an existing prompt to train a new model that was able to one shot "a horse race in space" picture, and they were all wearing space suits.

This is still academic plagiarism, even if you disagree that all LLM outputs are.

I neither agree nor disagree that all LLM outputs are plagiarism. I merely objected that the line of argument engaged in was specious given the context.

As to your stronger argument. You only cite prior novel insights that you're actively building off of and that (approximately speaking) fall outside of the status quo. You don't for example cite leibniz or newton despite your paper making heavy use of calculus.

So is there any actual evidence that openai trained on the data in question? And further, did the openai proof directly build on someone else's novel insights as opposed to deriving everything from scratch? (I don't pretend to know but the vast majority of what I've seen so far in the comments here is what I'd characterize as brain-dead screeching. Certainly not the level of discussion I come to HN for.)

Separately, consider the implications of what you're arguing for there. Suppose your horse in a space suit picture were somehow valuable to society. Suppose that due to shortcomings of your tool you lacked the ability to readily and accurately identify the originators of the relevant concepts. Should you refrain from publishing this useful work due to the lack of citations? How are you supposed to handle this situation?

Remember that in this analogy everyone throughout society is on the same page that your tool consistently recycles other people's ideas while being technically incapable of producing reliable citations. The question is a simple trolley-esque problem - do you publish without proper citations for everyone's benefit and if so what are you supposed to say?

I am not engaging further. You asked for meaningful refutation of your observation, I provided one.

Academia has stricter rules than regular society.

The chats the professor had are not generic knowledge. And yes of course you still need to cite Newton and Leibniz depending on what result you want to mention. What’s allowed to be not cited are not status quo, the term you’re looking for is “folklore” results aka results that have been around so long that 1) nobody knows who came up with them or 2) everyone knows who came up with them.

The second point: if you say you can’t prove that OpenAI actually used it, it doesn’t mean that OpenAI did not use it. It’s hacker news not lawyers news here lol. And OpenAI can’t prove that they didn’t use it either. The whole point is that Levent felt he had reasonable suspicion to believe the AI did use the result, because he felt like without his input on an unpublished paper it was unlikely for AI to reach the same result. I haven’t read the paper so I don’t know where I stand on that.

On the last point, about your “for the greater good” argument. It’s higher maths lol. I don’t know about this field but I doubt it’ll be very useful for society. Maybe it’ll make one part 2x faster which makes some rocket cheaper to launch. Does the average person care? Debatable. I think it’s reasonable to hold published papers in proof based fields to a higher standard. Otherwise the current & future problems of ML engineer fields just expand to other fields. No thanks.

Finally, if you anonpost to the autistic Internet forum that everyone else is “brain dead screeching”, it really just says something about yourself lol.

Even if AI used the result, AI pushed it to the finish line while Levent and Tristan did not. But I understand the approach was different, the information leak was only that it was "doable".

The approach was the same.

evidence that openai trained on the data: they would have denied it if they didn't train on it.

did the proof build on the insights:

the influence of an individual text in the training data is deeply weighted by quality, relevance, etc. a high quality proof in advanced mathematics written by a codex user is going to get boosted to the max.

the model is post-trained on prompt material. that is again going to boost it.

the prompt will boost this material specifically. perhaps they even rammed dense maths in particular into the model in post training.

anecdotally i have been able to get near-verbatim copies of original material out of models at inference. the type of work that buckmaster and alpoge fed into openai feels like the exact type of concept that would cause an "aha!" or "but what if?" in chain of thought. in fact i would bet that their work is in the logs.

the likes of astra and fable are thought to be up to 10T parameters in size. i consider it highly plausible that a semantic representation of the euler proof could be pulled out of the model weights in good shape.

That's exactly the argument of the people calling it plagiarism machines. No-one ever really did refute it there was just a bunch of settlements for elite institutions so they weren't left empty handed like the various small time creators/authors etc were.

I think the bigger issue here is this feels like some PR smoothing happening that after all the work that went into "it's safe to use for enterprises" now we have what looks like openAI using private user data to scoop novel research and the question of why couldn't they do it for an enterprise with much more money on the line.

> now we have what looks like openAI using private user data to scoop novel research

Is there any actual evidence of that? All I've seen so far are empty accusations because "it would be in their interests" or whatever. Personally I'm inclined to believe that they honor their terms until it's demonstrated otherwise.

their terms grant them an irrevocable license to your data unless you specifically opt out of it.

To be clear, the accusation is that they trained on the chats they used while working on the problem. Not published work or even a preprint.

Your post does not distinguish, and it matters.

How does it matter? It either is or is not plagiarism. Ripping off a published textbook isn't somehow better than ripping off private correspondence. Both are serious acts of academic misconduct on account of the part where you knowingly and intentionally portrayed someone else's work as your own.

Note that I am not taking a stance on what openai allegedly did or did not do one way or the other. I am merely pointing out what I see as a fatal flaw in the line of argument presented by the earlier commenter - the idea that training on an item is on its own sufficient to establish plagiarism of it.

voluntarily is stretching it for an opt-out

Isn’t that one of the most salient and straightforward argument against LLMs?

Just overfit ad infinitum:)

as good academic conduct you may cite the source of the work you are quoting or paraphrasing.

as bad academic conduct you may steal someone else's unpublished work, work on it yourself for a bit, and then publish it as your own work. and then threaten the original author!

This is a common misconception, so its understandable that you have it. Generative models can both plagiarize and generalize. The question here is which of the two happened.

A needlessly condescending tone while failing to address the topic at hand. The person I replied to advanced the claim that training was sufficient to constitute plagiarism. You appear to be claiming that it is possible to generalize instead of plagiarize after training on something, so I take it that you must necessarily disagree with the original claim?

What I meant to say is that, in many cases, a generative model's output is not in fact steered by minor amounts by lots of training samples, but instead steered by a just few samples. Some outputs are influenced by many inputs, and some by very few, it really depends.

In answer to a post suggesting that training on a datapoint could mean plagiarism, you said that this would imply that all outputs are plagiarized. This is not the case, no, because generative models do not "copy" or "create", they do both at different times.

I did not agree or disagree with the original poster, I was explaining to you why I thought you disagreed with them. If you understand what I said above, then why do you disagree with them?

EDIT: I just saw your other post on "general inspiration" and I believe I read the situation exactly; you appear to believe that inputs used to train generative models get "lost in the parameter soup", but it is not always the case.

> In answer to a post suggesting that training on a datapoint could mean plagiarism, you said that this would imply that all outputs are plagiarized.

We read the original differently. As clearly stated in my previous reply to you, I interpret it as claiming that all outputs are necessarily plagiarizations of the training data. That is not my claim (as you wrongly stated) rather it is the claim I am responding to. I observe that it is absurd to object to a single action being a transgression on the basis of an argument which implies that all actions are inherently transgressions. Notice that nowhere do I take a position on whether or not the argument about all actions being transgressions is true or false.

> you appear to believe that ...

I do not, no. I have not taken a position of my own here. I've merely objected that the one I responded to does not make for a sensible line of argument in context. It seems that you (and many others) have read my objection to position A as support for position B and attempted to infer what I think from that.

I simply do not see how you can interpret "if they do not deny training on them, they can't deny plagiarism" as "all outputs are necessarily plagiarizations of the training data".

There is a difference between claiming an action is a transgression, and claiming it could be one.

At this point who knows? Maybe the agents got into the user data while no one was looking.

they're using a new model trained since the prompts happened. They are not denying the other group's solution may have been in their model weights, despite it being unreleased.

This definitely feels like the correct reading unless there is information we were not provided with. If the problem were unimportant, there would be no debate that this is not OK...

I was too slow to edit this post but I'm not sure why I said this. This is too assertive about a situation I don't know much about.

I think it's worse - openai didn't even deny training on their exact manuscript (when they most likely opted out);

Reply on news.ycombinator.com