Hacker News Reader

Comment by burrish | Hacker News Reader

>While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.

What do you mean, as OpenAI employee, you cannot tell that his work has entered the training data ?

But also correct me if I'm wrong, if the two mathematician were really close to finish this problem, and their conversation were used by OpenAI, shouldn't the Agent have succeeded way faster/efficiently instead of using "4.9 million messages and used about 300 billion output tokens."

View on HN

I’m not a mathematician so take this with a grain of salt. Apparently terry tao commented that the approach used for the Euler paper can “probably” be used for solving NS but it’s still technically challenging and can probably be done with an LLM with a lot of compute. To me the crux of the issue is whether the insight were stolen so that the problem becomes something that is in the domain of LLMs. This is much different than LLMs coming up with the insight. OpenAI wants everyone to think the LLM came up with the insight and solved the thing by itself even though they have perhaps an army of researchers.

Having the chat logs enter the training data and having them have a meaningful influence on the ultimate result the model produces are very different things.

The text for all the Goosebumps books are certainly in the training data and to some small amount influenced the solve. But their contribution was so vanishingly small it would seem absurd to say R L Stein should have recourse for contibuting to the solve.

But this is different, right?

The equivalent would be taking a (fully offline) LLM and asking it about the ending of one specific Goosebumps book, and it revealing the twist. And although that specific book was (probably) only once in the training data, a high parameter LLM can usually "remember" the twist.

The only way it would be able to tell you the ending is if it was somehow given more importance in pretraining, loaded into context, or represented in multiple sets of training samples. I have a blog that I make very LLM friendly and usually load posts up into context when I’m working on something relevant. I’ve also opted to improve models for everyone. Despite this, the model can’t recognize my site or any of my posts when I ask it to recall without internet usage (I also turn memory off btw).

That's not correct for a SOTA model with trillions of parameters. Those have immense amounts of knowledge trained into their parameters. Try it. It "remembers" the ending of random books, Goosebumps and otherwise.

That's the entire point of why knowledge cutoff is so important (if you use them offline).

I find it entirely believable that the Navier-Stokes conversation was auto-flagged as high value training data, and burned-in the models knowlegdge base.

"shouldn't the Agent have succeeded way faster/efficiently instead of using "4.9 million messages and used about 300 billion output tokens."" No. Have you seen LLM output? The fact that it managed to converge at all in the problem space is amazing, an LLM is a 'book', the context window is the page number, the page has a token, that's it, that this huge book of weights already contains a method to search a problem space at all and actually go down a 'good' path eventually is remarkable, but it's going to be incredibly inefficient at doing so, a book doesn't have any kind of real-time model building like a brain.

It's almost as if it was actually found by manually written brute-force algorithm running on OpenAI's massive computer cluster.

Reply on news.ycombinator.com