Hacker News Reader

> They were coordinating with OpenAI regarding a publishing timeline, but could not come to an agreement,

Skimming the PDFs it seems much more dramatic than that? It sounds like at least one of them is concerned OpenAI "solved" the problem by having their internal model use the chats of the independent researchers and want to claim the credit instead? I don't know. The tone is pretty accusational though:

> the one Levent and I had quietly chosen to attack. Almost nobody else I know of was working on it. It is not the direction one arrives at in a few days by giving a model the problem statement. When I heard “forced,” it was a bright red flag.

> I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used.

> I asked when the first prompt had been sent by them. This question was not answered directly by OpenAI for some time. Eventually it was agreed that it had been sent in the past few days, after information about our work had reached OpenAI.

> I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer. [0]

[0]: https://cims.nyu.edu/~tristanb/statement.pdf

Yeah these are major accusations. But the story is incomplete, the conversation is missing a lot of details. It's not clear who was working on what, and when. The entire thing feels rushed, like they wanted to get this result published and out the door quickly.

Does he claim to have opted out of training too?

There are various forces at play here, academic honesty requires them to disclose any inputs regardless of license or ToS circumstances.

While common sense reminds us here that if you send your data to an external entity’s computer, you are no longer in control of said data. The lines have blurred here clearly over the last decade, but that should have made the theory yet more clear to everyone involved: your data will be vacuumed up unless you keep it sealed. Use your own computer if you want to be in control.

But if they didn't opt out of training, did they want OpenAI to opt out for them? Also they need to audit anyone they sent drafts to to make sure they opted out before submitting it.

I'd prefer things be opt in, and especially not start opt out, then try to trick you opt in with a popup defaulting to opt-in, like Anthropic did on consumer plans, but if they submitted anything on an opted-in plan it's not reasonable to be mad it trained on it.

Even still, I also believe for significant reasons that OpenAI would ignore the opt-out in selective cases and could be in the wrong here.

And the threats and terms they offered seem wrong either way, pending more context.

If you don't trust the other party, then it doesn't matter how the checkbox is set. The fundamental rule, IMO, is don't send precious or secret data to a third party.

I find the framing a little strange, a sort of David vs Goliath (with his enormous computational resources at his disposal). Since Levent is at Anthropic whose internal models are presumably as capable as anything OpenAI has. So why wasn't Anthropic behind their effort? Why did Tristan use OpenAI's models when it should have been known was a potential outcome? I understand they wanted a normal math collaboration but presumably what Levent brought was his resources (as far as I can see Navier-Stokes is not his speciality). Normally these things are hashed out formally beforehand to avoid the sort of thing now happening.

They were working on it for almost a year, and Buckmaster has evidently been interested in Navier-Stokes for a while. This seems to be more of an innocent collaboration between two researchers than a strong company PR effort. Maybe Anthropic should have stepped in and made a large team to help them finish the proof (and maybe they tried and didn't succeed, who knows).

If what he wrote is accurate, it does suggest that OAI is effectively extremely hostile to cutting edge researchers (eg, if we hear rumors about your partial success on a problem that has huge PR benefits, then we'll assemble a strike team of researchers with unlimited compute to claim the win for ourselves, possibly by training on your data). It's also not a good look for them to request author removals based on company affiliations.

I think what you have in mind is more appropriate for more normal corporate projects and the like. But academic collaborations are not usually so political/'profit' driven, if that makes sense.

> So why wasn't Anthropic behind their effort?

Presumably because this was something Levent did in his spare time and because it was not obvious that this work would eventually lead to a breakthrough.

> Why did Tristan use OpenAI's models when it should have been known was a potential outcome?

I'm sure in the past he had less cynical feelings about OpenAI and their penchant for academic fraud.

> I understand they wanted a normal math collaboration but presumably what Levent brought was his resources (as far as I can see Navier-Stokes is not his speciality)

I think you're not giving the guy enough credit in saying that his contribution came down to having an API key for Anthropic models.

> Normally these things are hashed out formally beforehand to avoid the sort of thing now happening.

How would that have helped? That agreement (which may well still exist) would not have involved OpenAI.

So what do you think his contribution was? His preprint record shows no research on fluids - and the statement says that the first LLM-generated proof Tristan received from Levent was 'the most horrendous I have ever read.' Levent is out for mathematical scalps whether it is in his field of expertise or not, and he has the resources to do it. And I am not saying he is not a very clever person, but the idea that you can bring yourself up to the forefront of research in PDEs, in particular NS, and contribute new ideas in less than a year is implausible.

They have messed things up, because Levent has a conflict of interest between his job at Anthropic and this independent work, and Tristan should have opted out of OpenAI training on their work (he probably didn't know about this). This doesn't justify OpenAI's despicable attempt to steal their work.

> A few days later, OpenAI gets back to him, and tells him an internal model found a counterexample for Navier–Stokes

Why is OpenAI chatting with him at all at this stage? Is the discussion along the lines of "hey we used the work you are famous for to do a bigger piece of work, just thought you should know" or "heyyy....so we kinda liked what you were typing in your private chat, and thought we'd develop those ideas a bit. and yeah we solved Navier-Stokes in the process. But it's our finding, so do you want like an honorary acknowledgement or do you want to go to court?"

Perhaps out of a sense of academic good will, knowing that he got there first?

It seems like the timeline according to OpenAI is that:

1. Buckmaster developed a counterexample to a reduced version of Navier-Stokes with Anthropic employee Levent

2. Rumors start spreading that Anthropic has solved Navier-Stokes

3. OpenAI learns this and starts throwing a ridiculous amount of compute at it, now knowing it's within reach of LLMs

4. Their LLMs (with human assistance) get FARTHER than Buckmaster, using the exact same method.

5. OpenAI reaches out to Buckmaster to negotiate a fair way to publish both results and properly assign credit

Perhaps they simply and honestly feel he is owed credit. I can't help but imagine it at least played some small part. I expect that's what Bubeck is going to claim: https://xcancel.com/SebastienBubeck/status/20972141224714323...

The only problem with this narrative is that they refused to allow the other coauthor to be listed because he worked at Anthropic.

That is absolutely *ridiculous* in academia to deny authorship because of affiliation of the author worked on a substantial portion. You’d be ostracized because nobody would ever want to work with you again.

>...has solved a millennium problem and is sitting on the result

Someone correct me if I'm wrong, but the work involved here is not the actual millennium problem, but it concerns versions with an added external force that the author thinks is a path that may help toward solving the harder unforced problem.

Apparently forcing is allowed in the Millenium Prize problem statement. So OpenAI's claimed proof could win the prize. OTOH the results Tristan and Levent are publishing here do not go far enough to win the prize, though apparently they are suggestive of a general approach that could produce a solution, which seems likely to be the general approach OpenAI's proof uses.

The question is whether OpenAI's pursuit of this direction happened spontaneously, or as a result of them learning about Tristan's work somehow. To be clear, while the tone of this post seems quite accusatory, Tristan does not claim to know for sure whether OpenAI unfairly benefited from his work. Sholto Douglas from Anthropic is also on record saying the suggestion that OpenAI used Tristan's codex transcripts somehow is extremely unlikely to be true[1], which I agree with, though it doesn't rule out them learning of Tristan's work some other way. I am sure OpenAI will have a statement out tomorrow clarifying their position.

[1] https://x.com/_sholtodouglas/status/2097218240397410733

Why do you find it to be extremely unlikely?

Because very few people actually have access to these logs, all access is monitored and recorded, and improper access will get you fired. It's not worth risking your job over something like this.

Not that I have strong reasons to think this is not true, but what reasons do we have to think it is? Has this been audited before?

There is almost zero risk to your job (quite the opposite, you might be richly rewarded!) if you're simply doing something here which the company wants done. (remember, billions and billions of dollars are at stake here! Do you really think there is no chance at all they would do it??)

> Because very few people actually have access to these logs, all access is monitored and recorded, and improper access will get you fired. It's not worth risking your job over something like this.

That's beyond naive. The money this would mean for OpenAI (and the money they've already spent)...

Okay suppose that you have a trillion dollar competitor salivating at the mouth to ruin your business, which is based on user privacy.

Why would you risk the trillions of dollars worth of business for the niche result of Navier-Stokes, which your average person cannot differentiate from a JEMS paper?

Because your competitor is likely doing the same thing, so they wont even try to call you out. Oh oh or because you're planning the largest IPO in history. The opinions of average people on the paper aren't going to be the ones reflected in the markets...

If there's one thing I'm absolutely confident in, it's that Sam Altman personally goes to great lengths ensuring that ethical standards are upheld at his company.

As modeless said, Sholto Douglas works for Anthropic!

So to be fair, if Anthropic is *also* doing this (quite likely!) then Sholto would have a very strong incentive to try and spin it as highly unlikely that any of the big AI labs are possibly doing this.

Winning a millennium prize is not worth the fallout of "we will steal your IP"

They train on chat logs unless opted out. This really isn't a conspiratorial claim requiring humans to decide to steal IP if true.

His prior work predating OpenAI's interest in the problem was ingested over the last year as he made progress and used for training.

Then, with a prompting nudge from OpenAI's team who acknowledged hearing about the direction "Anthropic" (his co-collaborator) had been pursuing, they're able to point their giant amount of compute towards a known promising path to a proof and crossing the finish line first.

That's fair, but the proofs are different and there was no active perusing of Tristan's approach.

This is literally the business model of LLM companies.

"Buckmaster and Alpöge think they have found a counterexample for Navier-Stokes, but the paper is not yet presentable. "

Are you sure about this? I'm far far from the area but it doesn't look like it to me on first viewing (hypo-dispersive seems like a sizable difference to me and not covered in the clay prize description)

Yes, it's a separate problem. That's a mistake in my post.

This reminds me of a conversation I had with an engineer at google when I was angrily saying "it's not end to end encryption if you get my emails and can train your models on them" to which he said "we try not to do that". What a statement!