Hacker News Reader

Comment by 20k | Hacker News Reader

>I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.

If you think these companies are not training on your prompts you are incredibly naive. These models were built by stealing and pirating literally everything they can get their hands on no matter the legality. AI companies are always very specific about what they're not doing - in a way that you can drive a truck through the loopholes

View on HN

OpenAI cannot give a definitive answer here, because it is genuinely unknowable if Buckmaster's data is in the training set.

OpenAI explicitly uses user feedback (the thumbs up or thumbs down ratings), as RLHF to train models. However, this feedback is anonymized and stripped of user identifiers. If Buckmaster ever used this feature, then that conversation would be anonymized, saved, and used for training, but not tied back to him.

They cannot issue a blanket denial (which people so desperately desire), and instead repeat that "it's very unlikely" (which pisses people off), because they cannot in good faith claim to have zero data at all.

it would absolutely be knowable if Buckmaster's data is in the training set if he or Alpoge were to provide logs to OAI.

the real tricky question would be: "to what extent does a year of work, put into Codex, towards an eventual proved forced blowup for the Euler equations go into training data and become an influence on agents told to attack NS?", and given that the first signal that OAI got to assign all firepower to NS rather than broadly spray all Millennium Prize + other high profile questions was a proven unforced blowup of the Euler equations after only 50 hours, I would say this extent seems likely to be decently high

Exactly, especially when the company can assign "blame" to the models themselves "ops, they just escaped our commands not to store user inputs..."

My guess is that if you opt out of your data being included then they honour that instruction, but I wouldn't bet my business on it.

Why would a company that used petabytes of text, images and audio without caring about ownership suddenly draw the line at a random person's chats?

It's equally naive to believe FUD spread on the internet, without evidence.

But I would love to see more informed insight/discussion on this.

I mean, its the researchers themselves talking about this

https://mastodon.social/@tristanbuckmaster/11723647135247030...

Reply on news.ycombinator.com