My Mental Model of AI Broke on September 8 | Hacker News Reader

My Mental Model of AI Broke on September 8

rough-ideas.bearblog.dev

32 pointsiamanatom71 comments

I think the author is not aware of how much the Millennium Prize proof was actually driven by a human working for years towards that problem. The AI did not solve this on its own. A world class mathematician prompted it towards the proof. In my view, this is still a human achievement, not an AI achievement.

If I design a bulldozer to push a five ton rock, did I push the rock or the bulldozer?

If the AI really did get the Millennium Prize, then why can't you get a Millennium Prize when you have access to the exact same model in ChatGPT?

This is copium. Humans worked on it, but they didn't come close to actually solving it. Even if you do the whole "the AI looked at material in its training data" thing, modern AI can make their own math data to train on with RLVF. This new model is legitimately on a different plane of existence from modern mathematicians.

Using the word "copium" is a "if you smelt it you dealt it" type of deal.

it should be able to solve some of the five left Millenium Prize problems then in a few weeks.

Why in weeks, not months or years?

Probably will solve a couple more at least in the coming year , no ? Why wouldn't it ?

My point is, the AI could not have done what it did without the skilled mathemetician prompting it. It was just a tool in the hands of the human that did it. And that was a very skilled human. Unless you're a professional mathematician, you cannot get the same result even if you use the same AI model. That's the proof that it's the human's work, not the AI.

Why don't you try getting a Millennium Prize then? If you think it was the AI that did it, you have access to the exact same ChatGPT.

I don't have access to the exact agent OpenAI used, and I certainly don't have access to the enormous amount of tokens that were used. That being said, I doubt there was some advanced prompting technique going on here. We've seen AI proofs with the prompts attached, and it's pretty basic stuff like "don't give up".

I think you're not aware of the specifics of this case. A mathematician worked on the prompts for over a year to get this proof. Look up the details.

We don't know what the prompts were though. Terence Tao showed of some of his, and they are indeed huge, with lots of context, things that might work, things he knows don't etc. What we do know about this, is that 10k agents worked together on this. The initial prompt, while probably still very relevant, would get diluted over time. We simply don't know the level of inolvement of humans in reaching this proof.

My point is if you're not a mathematician on the level of Terence Tao you wouldn't have been able to get this result.

Thus, AI cannot be credited with the result. Because it's not AI that did it, it's the human that used it as a tool to get the result.

You could say that about every scientific/mathematical breakthrough. Einstein's Special relativity depended heavily on Lorentz's work, the Michelson-Morley experiments, Maxwell's equations. Grigori Perelman, the only person to have solved a Millenium Prize problem, noted how his work was only possible due to Richard S. Hamilton's work on Ricci flow.

Most scientific breakthroughs are just the completing the last 5% of work already done, but that last 5% is very hard and still only happens very rarely. That an AI was able to synthesize all the work and bring it forward is evidence that AI can make novel progress on the same level as renown mathematicians.

Einstein literally stated he was not influenced by the Michelson-Morley experiments, but it's a common myth that he was.

"When Ι asked him how he had learned of the Michelson Morley experiment, he told me that he had become aware of it through writings of Η. Α. Lοrentz, but only after 1905 had it come to his attention! "Otherwise" he said, "I would have mentioned it in my paper!" indeed, Einstein's 1905 paper contains no mention of Μichelson's experiment or references to Lorentz's papers."

From https://physics.stackexchange.com/questions/89375/did-einste...

Correcting a professor at MIT (head of the department alas) on this erroneous belief during a grad school interview cost me admission as he insisted otherwise and wouldn't back down. Ironically, one of my college professors was interviewed on NPR a week later and confirmed what I had stated.

Ask me what I think of checked out, tenured academics. Go ahead...

That may be true, but Lorentz specifically cited the MM experiments and was influenced by them, and Einstein cited and was influenced by Lorentz, so he was influenced, just one level removed.

>https://link.springer.com/chapter/10.1007/978-3-663-19510-8_...

Which he only became aware of after the paper...

In this 1905 special relativity paper he mentions "the Lorentzian electrodynamics and optics of moving bodies"

https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf

And did you read the footnote on the first page?

"The preceding memoir by Lorentz was not at this time known to the author."

It may also have been a social test — ie the polite thing to do in that circumstance is to say “oh wow, I thought I knew that one, weird. I’ll have to look that up again thank you” even if you know 100% that you’re correct. Weird social expectation thing.

It was the 1980s and that very well may have been what was going on, but if that's the game they're going to play then I have better places to play games. Ironically, I got several apologies from other MIT professors a few years later who ended up working with me on various things.

> That an AI was able to synthesize all the work and bring it forward

But that's not what it did. A mathematician literally prompted it towards that, helping it every step of the way for years.

And only the mathematician was able to make the AI do that. Anybody else with the same AI could not do the same thing.

If you think it was the AI that solved a Millennium Prize, then why don't you try using ChatGPT to get a Millennium Prize?

>If you think it was the AI that solved a Millennium Prize, then why don't you try using ChatGPT to get a Millennium Prize?

The model which did the NS solution is not yet available to the public, and it required over 15 million dollars of tokens (based on API pricing) to produce the solution.

>And only the mathematician was able to make the AI do that. Anybody else with the same AI could not do the same thing.

That is only speculation, the total sum of the prompt's help to the model could have just been suggesting research directions. Also mathematicians talk to each other and use/learn from each others' research all the time. Should every mathematician, in order to produce a "legitimate" proof, lock themselves in a room for the entire duration of their work and ensure no one else helped?

> If I design a bulldozer to push a five ton rock, did I push the rock or the bulldozer?

Unequivocally the bulldozer. You get to take the blame in design of the bulldozer, though.

The law (which is supposed to codify the common sense and shared values underpinning a society—note "supposed to") disagrees with you: if I use a bulldozer to damage your property, you can sue me, not the bulldozer or its maker.

Sure, legally.

But practically speaking, who did the heavy lifting? The operator or the machine?

Everyone who's ideas made bulldozer possible, I would say..

No, no, you don't get it - soon Joe Sixpack will be prompting AGI "solve me {super difficult problem researchers couldn't solved for centuries}" and releasing their own research papers!

I guess this post is not worthy of the front page. It is poorly written, repetitive, and feels like it was written by a LLM. It seems more like a reactionary post about events that have already happened. There is no insightful signal whatsoever just a remix of existing rhetoric. Ironically, the blog is also called "Rough Ideas" and the homepage says that the blog may contain rough ideas. I guess Hacker News has declined in quality these days. I might get flagged for saying this.

The problem isn't the mental model of AI--it's the mental model of intelligence. If you think intelligence is some non-algorithmic, non-computable process then of course you won't believe that an AI can be intelligent.

But since Turing's time we've known that intelligence is just computation--it's not until recently that we've been able to come up with the specific algorithm.

Think back to Kasparov playing Deep Blue. Back then, some people (including Kasparov) believed that a computer would never beat a human. They felt that human creativity and ability to see the whole board would always beat brute-force computation.

I watched the pivotal game 5 live. There was a point where Deep Blue made a pawn move away from the main action. The commentators at the time, chess master all, almost cheered--it looked like the machine had blundered. "It's playing like a computer" they said. But one look at Kasparov told you they were wrong. Kasparov was worried. The main action resolved, but in the end, that one pawn move, 20 moves prior, left Deep Blue in a better position.

What modern LLMs do is apply brute-force computation to any domain expressible in language--not just a restricted chess domain. That's the algorithm.

> What modern LLMs do is apply brute-force computation to any domain expressible in language--not just a restricted chess domain. That's the algorithm.

Which means that companies with sufficient computational resources and money will be capable of unlocking problems thousands of times faster and more effective than any individual even when lacking the skills, just by a matter of try and error.

I'm continually perplexed by people's perception that AI would be incapable of generating new ideas or discoveries, even years ago.

Deterministic machines do the same stuff again and again. Add entropy and they do new original stuff. Add a checker or verifier and you can filter for new stuff that is better. At the very least here, you now have evolution.

There is nothing that is particularly compelling about a system that can generate new stuff that is an improvement. What's compelling if anything is the verifier, but that isn't particularly any more mysterious than LLM output already. At least not nearly as mysterious as "Meat brains have a magical ability to manifest original ideas".

>Meat brains have a magical ability to manifest original ideas..

No, no. Any random sentence generator can generate original idea. Actually it is said that randomness contain all the answers. You don't need a "meat brain" to do that.

> I'm continually perplexed by people's perception that AI would be incapable of generating new ideas or discoveries, even years ago.

It's still an incredibly common claim, at least on places like Reddit. Perhaps Doctrow has been pushing the idea or something?

And Zitron claimed that years ago AI was already as good as it was ever going to be - like Zitron, I imagine a lot of people haven't changed their opinions in recent years even as AI advanced.

For me it was a few years ago. I had seen a twitter comment referencing astrology in a spat with two black female musicians in the US. My preconception was to look up why that kind of superstition was prevalent in those circles.

The answer I got from chapgpt was essentially that it has very little to do with superstition and all to do with being able to use a language to talk about stuff while still not move outside cultural norms. More sort of a secret language where you can probe questions like if your boyfriend is violent, or if your friend is having an affair.

I think last year I saw some research on how the reading of tea leaves originated in the ottoman empire, it was remarkably similar. The point is that I learned something new that would have been extraordinarily hard to google, or even understand without putting some serious study into the subject.

> even understand without putting some serious study into the subject.

Then how can you possibly know that it's true?

I can’t, and it’s not really the point. The point is that I got exposed to a new way of looking at a phenomenon that I haven’t thought about before.

> My belief system was shattered the day the proof was announced.

I can't imagine having eyes and being able to hold the wrong belief regarding AI for so long. The fact that AI can surpass humans and make novel contributions to our civilization was obvious for me at least about a year earlier.

You need to have pretty messianic view of humans to believe otherwise.

To me, as soon as it was obvious that training had distilled and connected abstract concepts of increasing generality, it was only a matter of time. Almost any "new" idea can be decomposed into a combination of old component concepts.

It was evident in GPT-3.5

> but that an AI system may have produced new mathematical knowledge that humanity did not have before

From the expose in Terence Taos blog [1], it seems the difficulty of the Navier-Stokes counter example is a delicate balancing act between having a blow-up solution and a well behaved force field. And this involves a lot of technical arguments based on already existing ideas.

If this is true, then the achievement of the AI is rather to correctly navigating this balancing than inventing something completely new.

[1] https://terrytao.wordpress.com/2026/09/07/finite-time-blowup...

It's hard to say exactly how much credit Astra gets if its training contained the research notes of Levent Alpöge and Tristan Buckmaster. Surely it's impressive to generate the result even if working from their notes but it muddies the waters on its capabilities quite a bit.

From https://openai.com/index/navier-stokes-solution/:

> we cannot rule out that de-identified data derived from their usage of our products helped improve our models

The cognitive bias in humans to believe higher intelligence is unique to humans and even supernatural is very, very strong.

Do you have like concrete proof this is not the case? We do not. We haven’t yet met higher intelligences. You have no basis to call it a bias.

This, despite the ongoing social litigation of if this is even real?

I believe AI is quite capable in the right circumstances, but I'm not convinced "this" is the watershed moment.

I think the watershed moment was when it was proven that it can solve highschoolers maths olympiad problems at competitive level. These problems require complexity of thinking that is beyond what most humans have to deal with in their entire lifetime. When AI took that in stride it was obvious that sky is the limit and entirety of current human achivement is a milestone but in a sense of the one that the car passes while doing 60.

>entirety of current human achivement..

The right way to look at it is that LLMs help us to maximize the utility of the entirety of current human achievement/knowledge by discovering obscure connections in it.

So pretty much the same thing that humans do. New things come out of those connections.

Yea, it is a better search tool than humans, as computers always were....

The qualitative difference is that it searches semantically across unstructured data and is able to cleverly combine related searches into an answer.

Imagine SQL but the queries mostly write themselves and database is just entirety of human knowledge with no formalization. If you were to create such thing 7 years ago you'd say somebody expects a miracle out of you. I don't get why so many people have trouble recognizing it now as such.

Yes, it is a very efficient search tool for existing data..

Comming back to SQL analogy... When you make a view over existing data tables that calculates something novel did you extend the amount of existing data?

Because AIs are sort of like that. Creating views of existing data no human has ever came up with. Views that might unlock new even more surprising and useful views. They are sort of materialized views because we are writing down what LLMs came up with. So they are new data tables. Derived, true. But possibly novel and unlocking something else.

Sorry. I don't agree that it "create" views autonomously. A human has to create a view in a database.

In the case of LLM, a human has to create a "prompt" that the LLM just continue, based on the patterns humans has created in the past.

Yeah, I see you are set in your ways. Maybe in a year of two of experiening frontier AIs firsthand you'll come around.

Where exactly do I need "coming around" based on what I said here?

It took the author until Sept. 8, 2026, to realize LLMs are not just stochastic parrots? I'm glad they did, but I'm not sure that is worthy of the front page.

"I remember early systems struggling with something as simple as 2+2. Then, within just a few years, we went from that to systems achieving IMO gold-medal-level performance and now, assuming this proof is correct, to a Millennium Prize problem. That completely changes how I think about the trajectory".

How would Sept. 8 completely change how they think about the trajectory? Seems like there has been tremendous progress at all time.

It isn't hard to conceive of things that plateau so perhaps OP thought that the 'intelligence' underlying these models would reach some mark and then level off. If instead they just keep getting smarter/better, that can really impact the highest potential use that people can imagine for them.

They still are parrots. Just properly trained with a lot of data. Doesn’t make them not useful. But that’s what they are though.

I interpret the word 'parrot' to mean incapable of creative or original thought. Solving a major maths problem that has resisted the best mathematicians for so long seems to prove otherwise (even if it were just a matter of remixing old ideas, which is not the case here).

What do they need to do for you to consider them non parrots, and do you consider a lot of humans as parrots?

Did it create the path to the solution, or just grab it from conversations with a researcher working on the problem?

Good question. I think we will know very soon, perhaps not for this particular math problem, but they just need to solve another one independently and we'll know for sure. My bet is that they can.

> Solving a major maths problem that has resisted the best mathematicians for so long seems to prove otherwise (even if it were just a matter of remixing old ideas, which is not the case here).

I don't think we know enough about how they work to claim that. OpenAI said they had 10 THOUSANDS agents working on the problem, testing all ideas they found in the literature (including, it seems, the breakthrough of the guys who had it for the hypo viscose case).

Yup. They have much more bandwidth to test things, but no original thoughts (and tbh, not thoughts at all, actually).

>creative or original thought.

If this can only answer questions, then it fails this test. Because at least the question has to come from somewhere...

seems to me that many advancements so far are more the product of intelligent effort than pure brilliance, i'm still hopeful that human researchers (and humans in general) will remain better at asking the right questions and making good decisions

I don't know.

What's the tl;dr here?

"I'm one of the last few who needed convincing, now listen to my thoughts on what's next!"

Another convert who can parrot what the detractors said/are saying and that ends with a thoughts-and-prayers socially progressive umm hope this doesn’t exarcerbate hooman differences too much. See you don’t need an LLM to summarize.

The number of people I've heard calling AI a "stochastic parrot" is concerning. You don't understand what is or isn't stochastic and you're parroting this phrase. You yourself are a stochastic parrot.

> I had held a very different view of AI: that it was basically a stochastic parrot

imo it's been not like that for a few years now

(alternatively, stochastic parrots' abilities have been underestimated)

Are people paid to write this shite?

Yes: https://www.cnbc.com/2026/02/06/google-microsoft-pay-creator...

AI;dr

Yeah, people are paid to promote AI and people are paid to criticize AI. Regardless of the issue money flows to pundits on both sides. Then the manufactured engagement is milked for ad profit.

why should i care about your mental model of anything?

Reply on news.ycombinator.com