OpenAI fought dirty on career-making math problem
techcrunch.com
techcrunch.com
The “least worst” representation of the course of events is that their researchers are just jerks, and not behaving in a manner that’s considered acceptable in the research community.
Other alleged scenarios just go downhill from there.
They were surprised another competitor had not wastefully thrown $20 million+ worth of compute at a problem they had no business solving in the first place?
That's reaching, imho.
They didn't know what they were doing and they pumped a bunch more carbon into our atmosphere because hubris basically.
Shame on them.
1. This is not the first time they have done these “hey guys check out this breakthrough!” announcements where others quickly came along and say “hey, not so fast.” (Eg Erdos) So, specifically in maths their reputation is not good.
2. They’re blurring the lines between commercial cutthroat developments and the gentlemen’s code of sorts re what’s acceptable in academic research. They were working from material non-public insights into other research, which is why they even tried to poke at this in the way they did. Doing that without collaborating first was a pretty jerk move no mater how you slice it.
3. There’s still lots of open questions about how novel the solution was and the timing here where this “test” only happened after other researchers say they fed OpenAI models at least part of the solution is a little too convenient to gloss over. OpenAI statements here to date on the matter have been rather fuzzy.
All that combined with OpenAI’s less than stellar reputation on ethics is why folks are reacting the way they are right now.
And that is a good thing! Competition moves us forward much faster than sitting on results to avoid hurting someone's feelings.
Legit competition is of course good. But taking what was arguably a leak and then using that divert massive sums of compute (not available to the other researchers that were using their systems) to then get ahead is where it gets ethically dubious.
> However, our proofs differ significantly and even the precise results proved are different in the Euler case (forced vs unforced).
How true is this?
If the implied statement is true (i.e. OAI couldn't have stolen results because the results were so different anyways), then I feel like it's pretty clear that OAI solved the problem on their own, and offering some amount of credit to Tristan and Levent is generous. Though on the other hand it's dirty to have even attempted a scoop in the first place.
If the implied statement is false, and Tristan/Levent's results are a substantial portion of the solution to the millennium problem, then it's probably unintentional but clear plagiarism. I think it's plausible to assume that OAI has trained their models on Tristan's codex conversations, and so regardless of legal ownership, the academic ownership definitely includes Tristan and Levent.
The rest of the drama (individual statements and wordings, e.g. by Sebastien) seems like a bit of a red herring. Worth noting, but not worth basing conclusive judgements on regarding academic misconduct. Regardless, OAI does not seem like the good guys.
I expect a lot more of this kind of drama in the near future.
I am supposing it is https://en.wikipedia.org/wiki/S%C3%A9bastien_Bubeck
This is terrible:
When Buckmaster pushed to make the dispute public, he says that Bubeck replied: “Why would you ruin your career?” Buckmaster says that when he pushed back, Bubeck followed up with: “If you don’t want me to be nice, then I don’t have to be nice.”
Stains an important moment in the history of AI progress for me. The future seems bleak with people like this at the reins.
Terrence Tao recently published an interesting take that zooms out from the details of the Navier Stokes drama.
Its an interesting observation he makes, because it is not dissimilar from the relatively common phenomenon of one academic lab getting scooped by another lab (usually by coincidence).
I was with you up to “usually by coincidence”.
There’s a long and sordid history in areas of chemistry and areas of biology of holding up a competing paper in review so you can scoop them. I’m sure it exists in physics as well. Certainly biophysics, but probably most subfields.
Often it’s a famous labs that can steamroll review or even just dump the work into PNAS as a “member contribution.”
At least one author of a famous inorganic chemistry textbook was rumored to do this routinely.
And I know of at least one National Academy member who swore off arxiv prepublication after getting scooped.
None of this makes it all right. But plagiarism and academic theft is old and definitely not always accidental.
A.) The actual steps of the proof, which I assume the AI would provide. B.) People, while working toward a solution, finding novel properties/methods along the way that open new avenues of research + new open problems.
Does B.) actually happen? Would knowing the solution to a problem stymie the process of finding new open questions? I would assume finding a solution may unlock other problems too. So maybe on balance it's not really bad?
I guess in the end, I'm just making the obvious case "the future is uncertain in the face of AI".
Statement from OpenAI: https://news.ycombinator.com/item?id=49613262
I can read the 2 pages in ~30s or so. How long is the video? 15m? Nah.
Edit: video is 37 minutes
> All told, the week-long effort consumed 300 billion output tokens — $22.5 million worth of compute, if charged at current Astra rates.
> The Navier-Stokes existence and smoothness problem is one of the seven Millennium Prize problems — a set of major unsolved math problems, each carrying a $1 million bounty
I know openAI isn't solving these problems in order to make profit, but it's interesting to guess how close we are to these things becoming profitable. Eg, if you think their public pricing for Astra is ~2x as expensive as their internal price, then they lost ~10M on net for this proof. That's not profitable, but it is much better than I would have expected, which is exciting for the other Millennium prize problems! Of course the fundamental approach (which they may have plagiarized from Buckmaster and Alpoge) might have added cost to that as well. Nonetheless, I wouldn't be too surprised if they're all solved within the next 3 years!
This makes the 100 billion tokens (total in/out) I've spent on my project on the $200/mo plan seem like a deal. Wow.
Such a satisfying quote.
OpenAI has been behind in the AI race since last November. They have been playing catch-up.
They recently released Astra and declared it is AGI. Now they are desperate for anything they can use as evidence for that claim.
In other words: they had clear motive to do anything they could to steal the glory from prominent mathematicians who worked hard on this problem and solved it. And they also cannot prove that said mathematician's data was not accessed by either the model or the OAI users prompting the model.
Does it prove what it claims to prove? Were new mathematics invented? Can this be applied to other areas?
I don’t have a problem with labs solving hard problems if they can, but they could at least pretend to care more about the problems and less about the marketing opportunity.
> Regarding the level of human involvement on our end: although a group of people was involved in our efforts, we collectively had no research-level expertise in fluid dynamics and the Navier-Stokes problem, and therefore were unable to meaningfully contribute to the mathematical content.
Trust: https://x.com/__alpoge__
"since on my side things were mostly me and claude having a good time yoloing random stuff in the corner rather than anything institutional"
Not: https://x.com/SebastienBubeck
"If you don't want me to be nice, then I don't have to be nice." [when clearly there's not enough deference paid to where this line of research originated]
I had planned to say on announcing our work that the results are not the important thing. Rather the important thing is instead the significance that a mathematician and an LLM model can now do all this work in a month
Get ready, bag-holders on index-tracking funds... you're about to lose your shirts.
I can't wait to see your short position that will make you rich enough to retire.