Hacker News Reader

> I get the feeling that mathematicians are needlessly panicking because they don't really understand how AI works.

I sunno if mathematicians would be having problems understanding how matrix multiplication, backprop, sigmoid functions, attention, embedding distances, probabilities, etc work.

As a group, they are probably more likely to understand it than everyone else.

That's a bit like saying that a physicist is more like to understand how a car works than anyone else because they understand all the principles of an internal combustion engine. And yet, curiously, when we take our car to the garage the person fixing it does not tend to have a physics degree.

Wanna guess why? I'm too tired now to expand the argument properly but basically understanding the components of a complex system doesn't mean you understand the principles of the system. A mathematician who is not an expert in AI has no reason to be particularly capable of understanding how AI works, i.e. how all the maths that go into creating an AI system come together to create. An AI system.

> How many human-years were invested in solving the same problem, before they were overtaken in the last few days by an AI? 90? Not even: that's just the time since Jeal Leray's statement of the problem in 1934. 26, if you want to count the time since 2000 when the Clay Institute named it as one of its Millennium Prize problems.

I hope this isn't actually news to you, but: There is more than one human. There is even more than one mathematician.

If there happen to have been as many as four humans working on Navier-Stokes at any given time since the year 2000, then that's more human-years applied to the problem than agent-years.

> How many mathematicians have worked on the problem? 10k? Not likely.

You don't get to count the factor of 10k once when working out how many agent-years OpenAI gave to the problem and again when demanding that for parity there would need to have been 10k mathematicians on it.

> And all that's without even considering whether the AI based its proof on carelessly shared work by the humans. Or rather, yes, let's consider that: it totally did.

Let's suppose that indeed what Buckmaster and Alpöge had done was in the model's training data. Well, it didn't enable Buckmaster and Alpöge to solve the problem for Navier-Stokes (they could only do Euler), and it did enable OpenAI's model to do that.

Also: we don't actually know that what they'd done was in the training data; the latest bits of what they'd done that could plausibly have been in the training data were from before when Buckmaster said they progressed from preliminaries ("We worked through the literature and upgraded various preliminary results") to actually making substantial progress on the problem ("This was until about a month ago, when we had real progress"); and from what Buckmaster wrote it sure seems like a lot of the Buckmaster/Alpöge progress was in fact done by LLMs. (E.g., Buckmaster says that he and Alpöge have been working frantically to try to understand the proof for their Euler solution. That sounds to me much more like "an LLM did this thing" than "we figured out all the hard bits and the LLM did nothing more than filling in a few details".)

Buckmaster's own account of things is that all the really clever ideas were those of Córdoba and Martínez-Zoroa. (Which are already out there in the open literature, and there is nothing remotely improper about making use of them.) And my understanding (but, note, I am not an expert on fluid dynamics or PDEs and I could be wrong) is that actually the OpenAI model's construction is quite different from that of C&MZ. On what basis are you confident that "the AI based its proof on" what B&A did?

(For the avoidance of doubt: I am not arguing that what OpenAI did was OK. Even if they actually didn't train at all on any of the Buckmaster/Alpöge chats, it's very much not good professional ethics to hear that someone else is working on something and rush to try to scoop them, and there is absolutely no question that they did that. The question here is how impressed we should be by the model's mathematical prowess.)

> A success rate of 17%?

A success rate of 17% on problems of this difficulty and significance is something that for any human being would be a career-defining triumph.

> We have thousands of problems solved by humans in the last 2k years and how many by AI? A couple dozen? Oooh scary!

That would be a more convincing argument if the AIs, like the humans, had been around and trying to solve those problems for the last 2k years. However, as you might have noticed, the state of the art in AI was rather primitive 2000 years ago.

>> If there happen to have been as many as four humans working on Navier-Stokes at any given time since the year 2000, then that's more human-years applied to the problem than agent-years.

My bad for not showing my work and inadvertently leading you down the garden path, but the "~100 agent-years" calculation goes like this:

10,000 agents * 88 hours = 880,000 agent-hours

88,000 agent-hours / 24 hours = 36,666.7 agent-days

36,666.7 agent-days / 365 days = 100.5 agent-years.

That's what you get for working 24 hours a day, 7 days a week, 365 days a year. Realistically speaking, that's not a work schedule any human can follow.

It's hard to make a realistic estimate because normally even a very dedicated mathematician will not be working exclusively on one problem all their waking time, or even all their working time. But, let's ignore this and assume a pretty standard work schedule of 8 working hours, five working days a week, and 52 working weeks a year.

Now, that's:

8 hours * 5 days = 40 working hours a week

40 hours * 52 weeks a year = 2080 hours a year

880,000 agent-hours / 4 humans = 220,000 hours per human

220,000 hours per human / 2080 hours a year = ~105.8 years

To clarify, that's how I estimate the number of years it would take a mathematician to do a quarter of the work of the 10k OpenAI agents if that mathematician worked only on solving Navier-Stokes and did nothing else in their entire career.

That's just not a realistic work schedule for any human. You can adjust the working hours if you want but I don't believe you'll get any realistic estimate. Don't forget that most academics' careers last around 30 years from PhD to Professor Emeritus. If you want a more realistic estimate of how much time it would take how many humans to do the work of the 10k OpenAI agents, you can start from that assumption and work your way up from that.

>> That would be a more convincing argument if the AIs, like the humans, had been around and trying to solve those problems for the last 2k years. However, as you might have noticed, the state of the art in AI was rather primitive 2000 years ago.

Sure. But the thing is agents can run 24/7, 365/365 in parallel and as you see above they can cover 2000 years of human work in much less time. I'm not going to estimate how much because the only bottleneck is the amount of compute and money that an AI company wishes to spend, and that depends on their motivation to solve a particular problem. However, with sufficient motivation 2k years of human research (keeping mind that's not 2k years of continuous work) can be covered in a few ... months? Probably.

> My bad for not showing my work and inadvertently leading you down the garden path

The problem isn't that you didn't show your work, it's that your work was wrong.

I entirely agree with your calculation that 10k agents for 88 hours is about 100 agent-years if we assume 24/7/365 operation. That's not what I was disagreeing with.

But then you said "How many human-years ...?" followed by estimating not the number of human-years that have gone into the problem but merely the number of years.

You can compare elapsed years for humans (26) and elapsed years for AI systems (about 0.01). You can compare agent-years (about 100) and human-years (26 times the average number of humans working on Navier-Stokes at any given time). Either of those is defensible.

But it makes absolutely no sense at all to compare agent-years for the AIs and elapsed years for the humans. Which is what you did.

If a typical human mathematician works 2000 hours a year (actual human mathematicians generally find that they can't do 8 hours a day of focused hard intellectual work, but I think we should count some of their "percolation time" too) then that's about 6 human-years per mathematician. So to get the same amount of mathematician-work as agent-work the average number of mathematicians you need to have been on the job is about 100/6, or about 16.

So when you wrote

> How many mathematicians have worked on the problem? 10k? Not likely.

the 10k figure was a total irrelevance. The number it would actually have to have been is about 16.

(My earlier "as many as four" ignored the fact that humans don't work 24/7/365, as you point out. But my point is that however you slice it the relevant number is more like four than it is like 10,000.)

My guess, for what it's worth is that that is roughly the order of magnitude of the number of human mathematicians working primarily on things that could be classified as "trying to make progress toward resolving the Navier-Stokes problem" during that time. I wouldn't be surprised if the actual figure were 3x bigger or 3x smaller. It probably depends on how broadly you interpret "trying to make progress toward resolving the Navier-Stokes problem", and one important difference is that all those human mathematicians leave behind them a trail of papers proving things that, whether or not they end up on the path to Navier-Stokes, may turn out to be useful later, whereas if OpenAI's agent swarm proved a lot of useful theorems along the way most of them never got published.

I don't, of course, disagree that it's possible for an AI company to put a lot of AI agents to work on a problem, but I'm not sure how that makes what they can do less impressive. The fact that you can do that has always been a major part of why AI could be such a big deal. "A country of geniuses in a datacentre" is the kind of thing people have said; we aren't quite there yet, but the "country" part is as important as the "geniuses" part.

I'm sorry but I'm not sure I understand your argument. I think you're saying I'm comparing apples to oranges. I'm not: I'm comparing apples to apples and oranges to oranges. These are two different questions:

>> But how much time have human brains spent working on the problem in either of those time periods? How many mathematicians have worked on the problem? 10k?

So neither 10k humans worked on Navier-Stokes, nor has any human spent a century of non-stop work on it.

But I could have made the point more clear maybe.

>> I don't, of course, disagree that it's possible for an AI company to put a lot of AI agents to work on a problem, but I'm not sure how that makes what they can do less impressive. The fact that you can do that has always been a major part of why AI could be such a big deal. "A country of geniuses in a datacentre" is the kind of thing people have said; we aren't quite there yet, but the "country" part is as important as the "geniuses" part.

Yes, I see your point, but those are not geniuses. Grigori Perelman proved the Poincaré conjecture alone, though as he has emphasised his work was based on advances made by others, particularly Richard S. Hamilton. That we can call a genius: a single man who solves one of the most interesting problems in all of mathematics building on the work of his predecessors. 10k agents that search blindly and find a result by luck (or by stealing it), I don't agree we can call "genius". That's what I call "brute force". Anyone who wants to call OpenAI's agents "a country of geniuses" has first to deal with the fact that they look a lot like monkeys on typewriters.

(My apologies for the very slow response; I hadn't noticed your reply until just now. If YGQ doesn't reply to this, future readers should note that the long delay between their last comment and my reply is a sufficient explanation.)

You wrote, earlier:

> For the n'th time: the recent successes of AI in mathematics are the result of a brute-force attack. [...] ~100 GPU years. How many human-years were invested in solving the same problem, before they were overtaken in the last few days by an AI?

I don't see how this makes sense except as a comparison between total AI-agent effort and total human effort. And the result of that comparison is not that the AI agents have put dramatically more effort into attacking the problem than human mathematicians have. Any one of those 10k AI agents has spent a lot less time on Navier-Stokes than a human who's been working hard on it has. All of them collectively have spent less time on Navier-Stokes than the mathematical community at large.

The AIs put an amount of agent-time into the problem that's in something like the same ballpark as the amount of human-time the mathematical community has put in, and (I think) in fact clearly quite a bit smaller. The AIs got to the solution, which the humans hadn't. I don't claim that this makes the AIs more impressive than the humans -- it's plausible that humans were already substantially more than half-way there. But it means that the AIs are at least comparable to the humans when it comes to attacking this sort of problem.

I agree that it doesn't seem like OpenAI's models are as good at mathematics as Grigori Perelman. But a year ago they weren't as good at mathematics as me[1], and a year before that they weren't as good at mathematics as a typical university undergraduate, and a year before that they weren't as good at mathematics as a typical secondary school student. (Don't take any of those timings/levels too seriously; I haven't checked exactly how the progress went. But it's something along those lines.) How do you expect things to stand a year or two from now?

[1] I was an IMO silver medallist[2] and have a mathematics PhD from a good university, but I'm in my fifties and necessarily a bit rusty at this sort of thing, and while I spent a couple of years trying to be a pure mathematics researcher I wasn't terribly good at it and escaped to the Real World.

[2] For those who don't know how it works, this doesn't mean "second place"; they give out a lot of medals of each type. Roughly speaking the proportions nothing:bronze:silver:gold are 6:3:2:1, IIRC. I was a fairly mediocre silver medallist, so maybe somewhere around the 20th percentile of IMO team members worldwide.)

> they look at lot like monkeys on typewriters.

It is hard to take your arguments seriously when you say this sort of thing. 10k monkeys on typewriters, typing one letter per second per monkey, would take something like 300,000 (elapsed) years to get as far as typing "NAVIERSTOKES". (I am assuming typewriters with only letters, case-insensitive.)

The AIs made an amount of progress that was at least broadly comparable to the amount of progress humans had made, with a number of agent-hours of work that was not dramatically greater than the amount humans had taken, to complete the solution of the Navier-Stokes thing. So the AIs were, individually, in something like the same class of ability as the humans.

You may be as impressed or unimpressed by that as you please, of course. You may think it unwise to extrapolate superhuman performance in the future, perhaps e.g. on the grounds that the smarter we try to make them the less relevant training data there is from what humans have done before. You may absolutely have moral or legal or political objections to what the AI companies are doing or what you expect them to do in future. But whatever they are, these systems are a long long long way from being monkeys on typewriters now.

> the result of a brute-force attack

I would say "the result of easily scalable intelligent attack."