A misalignment of AI in mathematics
mathandai.org
mathandai.org
I am thinking of Mochizuki's abc conjecture: He worked in relative isolation, and dumped a huge incomprehensible proof on the community (to oversimplify a bit). That's not totally unlike what might happen if AI generates a huge, incomprehensible proof of let's say RH.
Well, what is the result? In the Mochizuki case, it was a lot of skepticism, but it also generated conferences, papers, talks in the hallway, discussions with students, and so on--a flurry of exactly that kind of community process that the declaration says is the main driver of mathematics.
Ultimately we think a fatal flaw was found in Mochizuki's proof, so it didn't lead anywhere in particular. But in our hypothetical "AI lean-verified proof of RH" situation, it would presumably generate substantially more of that community activity we saw in the Mochizuki situation. And if it's correct, that community activity would be productive (expository talks, students given problems to flesh out or generalize, etc).
Maybe mathematics just becomes a little more like other fields--relying on labs with lots of money for compute, digging through a corpus of AI-generated proofs, etc.
Dr. Tao said the same thing. Somehow, this letter came through. He wants to conduct Math competitions where participants who don’t have formal credentials can contribute to mathematical research through AI.
Title: Terence Tao - SAIR Competitions and the Future of Experimental Mathematics
https://www.youtube.com/watch?v=rB9YOi3lb7w
and this:
Daniel Litt - Working with LLMs to do high quality math
Apparently he has since changed his mind.
He doesn't see value in scrolling through unsolved problems asking an AI to please solve them. In his view, this is a fundamental confusion about what mathematical research is for. Knocking down unsolved problems without developing the community's understanding of them is like prompting Claude to go through a Jira board, write code for all the open tickets, and then close them without merging or deploying the code.
(A) it furthers human knowledge
(B) it gets used in applied sciences, engineering, etc.
Yet that's exactly how the field works. A new grad student is tasked with finding a suitably difficult problem from a list of unsolved problems. The sweet spot is obscure, so that fewer people are working on it, but not too obscure that no one knows about it. It works the same way in theoretical physics and theoretical Comp Sci, and I speak from insider knowledge. The rosy view of mathematicians in the media is largely a product of marketing.
I understand the perspective: The journey of a PhD thesis is a learning experience greatly beneficial to the student. Yet that journey is funded by society (esp. for domestic students) and society benefits from the results. The average person benefits when progress is made.
Most math textbooks have solutions in the back. That didn't wreck peoples ability to learn math, did it?
So we are left to speculate about why solutions in appendix are fine, but LLM solving open problems is not fine.
The idea that when LLMs produce solutions, people won’t try to understand them and won’t learn from it, is obviously not true. Terry Tao himself spent time digesting and simplifying LLM proofs.
So again we’re left to speculate what the actual problem is.
Math understanding will increase with LLMs. Not just professional mathematicians but amateurs.
I think you’re struggling to understand what Tao and his cosignatories are saying because you’ve acquired a very specific kind of “AI-pilled” mindset from social media, where taking AI seriously implies accepting LLMs should be used at any time for any purpose. They’re saying in great detail that LLM solutions are unhelpful when presented in a particular way, but you can’t help but hear them saying that LLM solutions aren’t helpful at all, even as you rightly point out that this makes no sense and is inconsistent with their observed behavior.
Society doesn't benefit from results in research mathematics because it mostly consists of pure mathematics, which is completely useless for society.
Thank You!
And embarrassingly they used him for a "coal miners should learn math" moment that just benefits the AI industry.
He has severely reversed course in the past week. Without concrete propositions it remains to be seen how much of the new resistance is for show.
AI is a tool. It speaks languages I don't (Math, Science, Code). I would love to participate in a Math competition without a hint of any formal advanced math training because my experience so far tells me I will do well.
I wrote a fancy polygon decomposition algorithm in university (pre-AI) which my professor didn't seem very impressed by because it was missing some sort of mathematical rigor. Yet everything I threw at it worked! Even he couldn't find a counter example.
It took a while for me to find some failing cases but it turned out they did exist.
But hey, maybe all I was missing is an AI-written lean proof.
Professor Tao.
It would be more like Lem's novel where it completely disappears from the human horizon: https://en.wikipedia.org/wiki/Golem_XIV
In the second case all human-level maths would be solved and what lies beyond would be always out of our scope.
Perhaps indeed a better understanding of what intelligence really is would allow this sort of Uplift (as in Brin's books).
if a particularly intelligent sixth grader was highly motivated to understand chromatic homotopy and had a highly capable private teacher available to her 24/7, she might within a year get to the point where she could apply it by herself to figure out some simple but nontrivial topological properties. (nobody has tested this. maybe it would require at least three years instead of one.)
if a particularly intelligent chimpanzee was for some reason highly motivated to understand chromatic homotopy, no matter how many years the most incredible teachers spent explaining it to her, she would never comprehend anything about it.
that's the pessimistic scenario: humans will be to future AI like our closest evolutionary relatives are to us. or even more pessimistic: we will be to future AI like insects are to us.
Is there an established term for the idea of "DoS"? I've taken to calling it slop fatigue.
This is just ticking a checkbox. And even worse and more time wasting even: you have ZERO proof that there's no latent Lean bug. Especially in a proof this large.
Maybe. But they're in 32 millions lines of Lean. How do you find the needle in the haystack ?
>Or, you could prompt agents later to analyze which lemmas or parts of the proof are surprising or applicable to other problems?
If OpenAI was truly serious about improving maths (and not jerking themselves off), they'd have also used Prove2Me (and contributed their results back), which would have done that. Each part of the proof combines into a larger graph, that everyone can reuse. Note that Anthropic isn't better there: yes, they used Prove2Me, but as far as I know they haven't contributed back to it, and just shat out 10 million lines and a good luck everyone.
Lee Sedol said in an interview that "losing to AI, in a sense, meant my entire world was collapsing. ... I could no longer enjoy the game. So I retired", and I think there will be folks in the mathematical community who would feel the same when the solutions pages to hard problems are suddenly available.
But on the other hand, people learned a lot from chess engines. After decades of chess computers beating humans, there was still a renewed interest in watching Leela beat Stockfish, with many people trying to understand the strategy Leela used.
If your happiness comes from grinding on a problem and making progress, the prospect of having to dig through a corpus of AI-generated proofs might be hard to swallow. But if you're willing to do that, you will still find beautiful things that only so many people can truly appreciate.
Carlsen is bored by studying engine lines.
The popularity is boosted by YouTubers because chess is very suitable for somewhat higher class content.
I'm not sure we'd want that world for math. Positions will be cut just like archaeologist positions are cut now.
If your goals are understanding the game, self improvement, building thinking skills-- this is the best chess has ever been. It's only if your goal is to beat every opponent you can find that chess is in a bad place.
Even if you don't blunder anything, you'll still find yourself in a worse position without any clue as of what went wrong and why.
Whereas when playing humans, they can usually explain their approach and when they noticed errors in your play.
It's the same for Magnus Carlsen. Even with Queen odds, Stockfish is literally unbeatable for the best players in the world. It's just too strong at evaluating all kinds of random tangent moves (and ensuing positional advantage) which no human player can possibly pay attention due to the time required.
Stockfish vs any human is like Carlsen vs other players by about 3-5 orders of magnitude[0]. It's that stark.
[0] A wild pun appears.
EDIT: To avoid having to respond to each responder, fair comments about Queen odds. Maybe I was thinking Rook odds? Also, I kinda lumped Stockfish in with all the other engines, but I realize there are other engines with different properties ofc.
But sometimes, these GMs can flag it, which counts as a win (especially when it's proxied by a cheater). Sometimes they can also explain the idea that cost them the game, so they've learned something maybe.
Whereas us scrubs literally cannot do anything at all for reasons completely beyond our understanding.
Computer moves are typically much more concrete than human moves: a human will play based on pattern matching ("intuition") and can only make explicit calculation of a small fraction of possibilities, after which decisions are guided by guesswork. The computers are unbeatable in practice because they can calculate concretely in seconds what might take an expert human long intensive study to notice, and they don't make the same kinds of oversights humans can make.
But if you stop and explore a particular position for an extended time, and if you have an intermediate level of chess skill, you too can probably often (usually?) figure out why it's doing something. Sometimes understanding the computer's reasons takes searching multiple branches of a tree several unlikely looking moves deep, but the collection of threats the computer was preemptively thwarting, traps it was setting, etc. are comprehensible to humans with enough effort, especially in games between the computer and a human.
The frustrating thing about playing against the computer is that it notices and thwarts every plan you might come up with, before you make up the plan yourself, and it doesn't make (human-apparent) mistakes, so the game ends up feeling hopeless. Nothing you try works on it, and if your idea is even slightly inaccurate it will be exploited.
Leela odds networks, on the other hand, are an entirely different beast. I cannot beat Leela queen odds, much less rook or minor piece odds, and even GMs struggle against Leela knight odds.
Without odds though, yeah, Stockfish is just incomprehensibly strong by human standards. All top chess engines are, but Stockfish moreso.
I'm not sure what the equivalent would look like in the math field, but it probably involves a lot of mathematicians losing their jobs and the quality of human-produced math decreasing overall.
The quality of the math in general would be fine, since in this scenario cpus will keep producing it. The quality of cpu-cpu chess games is quite high, beyond human understanding in many cases.
Chess is a weird example because it doesn't really have any utility beyond itself. Even pure math sometimes ends up having use in the strangest places. Although if no one understands the frontier math (because no one is getting paid to), I'm not sure it even matters what the quality of the cpu math is?
It's a bit like a tree falling in a forest. If an LLM proves a theorem but no one understands it, did it make a sound?
But in future most proofs will be for consumption by other AI models in the pursuit of yet other proofs.
It's kind of surprising so many mathematicians act surprised by this given this was clearly where automated proof assistants would lead. I guess they assumed they'd always be the ones guiding them.
What is the purpose of that?
Its like art being produced for AI to consume. What is gained from that?
But if AI is to recursively self improve understanding and evolving its own foundations, which are clearly mathematical, is essential. There is no need for humans to grasp what is going on in that loop.
Yes.
Friends and I often work on Putnam problems and this series:
The (Almost) Impossible Integrals, Sums, and Series by Cornel Ioan Vălean
But it's far from the only reason people in general do and have done math.
And at some point AI will start suggesting - or doing - physical experiments.
Does your "one" only contain humans or does it also contain other AI systems. AI math is not a single monolithic thing, but a distributed one. I see value in sharing proofs even among just AI.
There's two points about this I am assuming 1) Mathematics actually has a significant subjectivity to it and is community oriented and not just climbing a never ending list of theorems that exists in the universe 2) A lot of mathematical research work is inside of a subfield and isn't directly motivated by applications. Sometimes it is but e.g. people don't work on obscure theorems about elliptic curves because of a dire need for that but more because the community found it interesting.
Presumably AI will be connect the dots to the applications. As the declaration says, this isn't just about math. Human understanding is losing economic value. You can understand stuff on your own time, I guess.
The standard justification for pure math to holders of purse-strings is something like "it might lead to a useful application down the road, like crypto, who knows". That looks pretty inefficient now. We have to entertain the possibility that AI can develop the math needed for any application we put to it. Eg if number theory didn't exist, we could have asked AI for a way to transit messages securely and it would maybe come up with fermats little theorem as part of its solution or maybe come up with an approach we can't conceive of right now seeing as most of us are constrained to available number theory. Like how in the last year when I give an LLM a programming project I see it doesnt even bother with of the many software libraries I and others have written and just codes up the calls it needs on the fly or finds some other ad hoc solution.
Also I'd argue that chess's utility is ultimately the same as pure math, particularly in esoteric fields. These things are highly unlikely to ever lead to any sort of real world breakthrough or application. The main benefit is an outlet for human logic, creativity, and exploration - which significant self improvement possible along the journey for players.
Though I think even that's probably too socially utilitarian. I think ultimately the 'real' drive is the same in both fields - it's fun and personally rewarding.
That's fine, and no amount of machine excellence will keep you from enjoying recreational chess or recreational math.
There was a renaissance during Covid and due to 'The Queen's Gambit' where it gained much more mainstream popularity, but... Chess AI was already far far (like 1000+ Elo) ahead of human players at that point.
The thing is... chess is humans playing (communicating) with humans and that's what keeps it interesting. Check out the view counts of chess AI tourneys vs. human tourneys.
There is a real, undeniable possibility of AI becoming better at mathematics in the same way that it became better at chess and Go, and in such a scenario, one may expect the community's response to be comparable.
Stockfish does not steal research or scoop researchers.
The concentration of computing resources and capital should be examined by the math community.
Frames could literally cost 1 dollar (only happened after china entered this market), but good luck finding ones like this with good lenses.
You need to buy from one supplier who intentionally offers cheapest frames for 50+ dollars and those frames look like crap. The ones that look better (even if same plastic) cost 500+ dollars - and all due to price gouging.
For lenses I am not sure, but suspect something similar.
Fielmann in Germany has been offering very reasonably priced glasses and frames for ages.
- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.
- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.
I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.
And a lot of people are using trailing edge models just fine already.
> [...] but the level of quality that is considered essential for work will always stay pinned at "near-frontier".
Why? When we'll finally all write our software in Lean and prove it correct and prove it fast, it won't matter that a slightly more clever model could have found a slightly nicer proof or whatever.
Just like today people happily use Python for many programs, even though rewriting in C might give you a performance boost. Good enough is often good enough.
If that's your only objection: in a few years you can prove Rieman's hypothesis on your smartphone, no need for any trillionaires to give you permission. Does that make any change to your argument, or did it not actually matter?
For the latter: I assume that having a whole data centre will always be an advantage. I am saying that for a fixed target, like proving the Rieman hypothesis from scratch, the required hardware will shrink.
And, yes, the Rieman hypothesis hasn't been proven yet. So to take your fears into account, replace my example with something they've already done, like constructing a solution to the Navier-Stokes-problem.
Yes, but to what point the hardware will shrink? There are several orders of magnitude of difference in what in your mind AI will become and what more conservative people believe. You take your view as granted...
Tech companies are as much the topic of this post as AI, I think that's the immediacy.
There's a single answer to that question: yes, math very much has an impact without humans in the loop.
Math has a lot of applications, and those applications don't care whether eg the new faster matrix multiplication algorithm was found and proven correct by a machine or a meatbag.
You give your AI the problem, like 'make this computer program faster and more reliable', and it'll go off and do the math necessary to make it so.
For all we know, P vs NP will never be resolved neither by human nor AI, and the world will just keep turning.
I am saying that applications supplied and supply an inexhaustible amount of good problems and questions to consider. Purely theoretical concerns also supply some questions, but even if that well dries up for some reason, applications persist.
As the US has offshored manufacturing, the number of patents issued for those processes has fallen. Innovation occurs where the foundational understanding is applied; they arise from a desire to do the required work more efficiently.
Similarly in math, attempting to solve a problem leads to new questions. IF you actually do the work.
You need to have done enough of the work to know what the correct next question are, or you need to rely on AI for everything.
The main situation I can think of where better calculations have a really visible effect is video and image compression, and that stuff is very far away from mathematical proof territory.
t. low information voter.
If you tell someone they can't use a matrix multiplication with better asymptotic performance than n^2.3755 from 1990 they're going to shrug and not care.
I’ve watched quite a lot of YouTube videos where two machines compete, so you may not be completely right here
Top chess engine championship is pretty fun to watch.
In other videos he's called out the influence that this and similar games have had on human players in recent years, particularly around square denial and thorn pawn strategies.
There is no bound on the amibitions of AI. AI is set to replace anything done by people, and there won't be any room left for people. There isn't any task done by humans that AI won't be better at.
This is not a tenable outcome.
We should never have built machines with agency, rather than optimization processes that operate as subroutines of humans.
We haven't quite crossed that bridge, but we do appear to be standing on it.
Do you really think making labor obsolete and giving humans back their time is such a bad thing?
In the short term, AI is disempowering the vast majority of people in favor of a very small subset. In the also way too short term, AI is disempowering all people.
I believe we will distribute the fruits of AI's labor at the very least to the extend where everyone can live comfortably.
No one has an interest in putting large parts of society on the streets in some sort of dystopia.
Historically though, technological improvement has lead to large increases of living standards for the overwhelming majority of people, so I think that past trends support the utopian view more than the doomer view.
I am not assuming humans don't remain in control, but rather, I am not axiomatically assuming there's any inherent reason we will.
We have precedents of people warning about existential risks in the past, when the warnings turned out to be false, or overblown. In some cases, such overblown warnings led to serious negatives for society (demonization of nuclear power).
Those counterfactuals are important, though. We do have a history of averted disasters, albeit not as large. Ozone hole, Y2K, think about things that seemed overblown at the time, and consider whether they were actually overblown or whether there was a concerted effort to successfully avert them.
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
Wow.
Only on HN.
Also apparently not from the US? Or Russia?
This does not reflect what I see in my government.
I'd guess horses today have on average a better life than back then.
I am much more than my job. If I receive basic income, please replace my job ASAP!
And if I don't receive a basic income in a world where AI is capable of replacing all human jobs, then the problem is not AI, but the powers that be.
And if I don't receive enough, then again: the problem is not AI, but powers that be. And there are various solutions for that... and none of them are helped by me being anti-AI.
The limit is inflation.
Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
That's not necessarily true. Technology gives more power to everyone. The rise of factories historically did not give more power to the powers that were at the time (the aristocracy), but instead lifted the masses from poverty (after some initial turbulent period).
> Theyre never going to use that power to provide basic income for everybody. You cannot wave a wand to make them do that. Theyre going to use that power to accumulate even more resources and raw materials and impoverish/expel the "useless" labor.
Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths that abuse people just for fun. The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse and exploitation of other humans, but simply indifference.
> Realistically our leverage over the powers that be is mostly about our labor and our ability to withdraw it.
That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich. You are applying capitalist and socialist analysis to a system that transcends those terms (AI post-scarcity economy).
Did not give more power to everyone. At best you could say that it shifted power from landed gentry to industrialists. Even that switchover was less of a change than you'd think.
The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
The most brutal two wars in human history were that brutal precisely because industrialization made it possible.
>instead lifted the masses from poverty (after some initial turbulent period).
What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
The whole of that is glossed over by right wing libertarian propaganda but that last part is particularly underemphasized.
>Contrary to leftist propaganda, the rich are not some cartoon villains and psychopaths
Do you look at peter thiel and elon musk or the robber barons and see anything else?
The few billionaires ive encountered personally were no less sociopathic but they kept it hidden better. Power corrupts. Immense wealth corrupts. That isnt a leftist thing, that's a human thing.
>The reason why the rich currently exploit the poor is because doing so provides them with significant material gains. Once they can obtain the same or even better material gains by "exploiting" robotic labor instead of human labor, the logical outcome is not further abuse
False. It just shifts the focus of their exploitation from human labor to natural resources.
They'll fight over oil and minerals and gas and water resources and at best leave us to rot (homelessness will skyrocket) and at worst they'll find some excuse to exterminate those of us they particularly dislike (Gaza serves as a model here).
The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
>That is (partly) true, but only in the current economic system. Widespread human-level AI changes the equation, and not necessarily in favor of those who are currently rich.
Human level AI (assuming it ever happens) will simply make the fight over natural resources that much more intense because labor will matter that much less.
You're living at the tail end of a relatively golden period in a country where labor was the economic bottleneck and natural resources were relatively plentiful. Venezuela and Angola and Iraq are models of what happens when that equation is reversed.
Nobody gives a shit about appeasing the people who live in those countries, their labor is virtually worthless. They are a model for how the rest of us will be treated in a world where human labor loses its value.
Compare the standard of living in 1850 vs. 1950. Even of relatively poor people. I rest my case. Technology is the main force that improves human wellbeing. There are of course also other factors, but they are less important.
> The practical upshot of the beginning of industrialization was very negative. The enclosure movement stripped families of their land to push them to work in the factories where they would never go willingly. The famines in Ireland and Ukraine were both triggered by redirecting grain to export in order to fund domestic industrial expansion.
Yes, that's what I mean by "initial turbulent period". Perhaps the same will happen with AI, but it will be worth it in the end. Don't give up prematurely!
> What lifted the masses from poverty wasnt the factories it was the labor movement which occurred in response to horrific working conditions and the reliance those factory workers had on mass labor (specifically coal mining which was incredibly labor intensive and a key economic chokepoint).
It was not one or the other. It was both. The rise from poverty would not be possible if factories were not developed. And I am not saying that in the AI world we would not have to fight for our rights. Of course we would. But the problem is not AI, just like historically the problem were not the actual machines in factories.
> False. It just shifts the focus of their exploitation from human labor to natural resources.
Exploiting more natural resources is the only way to increase standard of living of humanity. I am OK with that. Resources don't have feelings, and ecology is not more important than human wellbeing.
> The world economy's reliance on human labor has been the best inducement to peace there is. It's no coincidence that all of the countries in the world with lots of natural resources and no industry are the biggest shitholes and vice versa.
The reason why some countries become shitholes is mostly ideological (extremist political or religious ideologies take hold of the population). Every shithole country is non-democratic (communist, totalitarian, fascist, theocratic etc..). This is not a problem of natural resources. It is a problem of people, their education, their beliefs/ideology, or, as capitalists say, "human capital" is the main problem here. AI could help here too, especially with education.
But yes, if people themselves are largely ignorant and extremist, no amount of resources and human-level AI robots will help them make a well-functioning society. You could drop masses of AGI robots into Afghanistan tomorrow, and people will just use them to kill or oppress each other more effectively, instead of using them to start building an AGI utopia...
Chess is only valuable as an entertainment. So no one really gains anything from computers becoming really good at chess.
But with math, everyone in the world would gain from computers becoming really, really good at it.
There's lots of problems like that. Eg if we prove P != NP, that won't have much of an immediate effect either.
However, there's also plenty of problems whose solutions will have practical effects, some even immediate.
Sure. But do you know which ones they are? Or do we discover later that they were valuable?
Your argument would be 100x more convincing if you gave an example.
I will try: a super-compressor that made my 100Mb web app into a 5 kb binary bundle would immediately speed up my work. Can/will AI move human understanding or machine capabilities on this front?
A browser without security vulnerabilities would be wonderful. I think LLMs are already helping with this a lot, but a lot of complexity remains.
A right to privacy in society would be amazing (see the UN Declararion of Human Rights). AI is eroding this.
So I tried but I’m not very impressed with my list. Do you have one?
This has nothing to do with mathematics.
> So I tried but I’m not very impressed with my list. Do you have one?
Look into operations research. Or narrower, you can look at improvements in linear programming solvers and mixed integer linear programming.
(These are examples of areas that have seen mathematical improvements in applications recently. I don't think good AI has been around for long enough to contribute much to progress there, yet.)
Take the Navier-Stoke problem for example. Knowing that there are solutions that "blow up" probably doesn't have a lot of practical applications. Such solutions couldn't happen in a real system. But the process of finding that proof could result in increased understanding of how turbulence works, or new techniques for solving non-linear partial differential equations (which has a lot of applications in science and engineering).
Sure. And AIs can use ideas from AI published proofs in one domain to inspire other domains just fine. Nothing changes here.
AI’s solutions are like the answers section to practice problems at the back of a textbook. Answer is 42, so what?You have to attempt the problem yourself, that’s the whole point of the exercise.
As an engineer I’m happy to use AI for math. If I publish a paper that way, very common these days, I think it’s still problematic. My paper would include something I didn’t come up with and I don’t really understand.
I think this is a good time to properly discuss these things because AI is coming for everything. Mathematics and Software were just the first two.
Yes, you might not get new human capabilities. But your applications still work better.
I think the error you're making is that you're assuming these systems have the same mathematical capability as humans (or better). But that's not the case, nor would an informed prediction be that they surely will get there soon enough if technological evolution keeps apace. That would be akin to believing a hiker will reach the moon if they will just keep ascending the mountain. "But look, they are making such good progress!"
I'm not convinced of that.
It is one possible outcome but there are also other possible outcomes.
What makes you think these companies (and I'm not a fan of all their motives) are not interested in developing research? The motives may be self-serving, but it is undoubtedly and objectively accelerating research.
Part of the point of the letter is that it is quite possible to act in a way that is a net negative to research. The most obvious case is when the companies violate ethical standards in research.
The subtler case, the one for maths in particular, is what happens when you fail to follow well-established patterns for making maths research productive. Tao himself spelled out how that can look in https://mathstodon.xyz/@tao/117207856734787448 (which notably came before any of the news on Navier–Stokes).
Sorry, but this is backwards. Mathematicians are the ones coming to the table asking for money from the taxpayer. This is the context we are speaking in. Mathematicians speaking with the taxpayers who fund them. It's not a great strategy to get offended or speak from a high horse. If the taxpayer is not getting it, patiently explain why you think this art project should remain funded at the same level instead of spending it on something else.
It'll basically become slop fatigue if OpenAI starts dumping out proofs faster than the community can keep up, and some turn out to be wrong, never formalize it, don't stay to support it, etc.
(Well that's my hopeful, optimistic take, anyway.)
It would also be a considerably more impressive achievement, because experts had mostly shifted to Navier-Stokes regularity being false, while as far as I know almost everybody thinks BSD is true. Hodge people seem less sure about.
If either conjecture is true and they prove it, that would be an even bigger success, since the techniques might unlock any number of other theorems.
That said, that’s probably just because of the drama miring their most recent one. After 2 I don’t see why they’d bother anymore.
P/NP and the Riemann Hypothesis are part of the milllenium problems. They will 100% keep trying to crack those regardless.
Mochizuki was still one human and it required legions of other humans to unpack and untangle to confirm that it didn't lead to anywhere in particular.
AI is now capable of constructions so complex that no human or human team can unpack. And its ability to increase that complexity is growing while our human ability is stagnant.
meta-AI analysis cannot help. We (software professionals who use AI regularly) already know that if you run into a situation where a Fable/Astra-generated analysis reaches the limits of our comprehension/complexity due to their subjectivity, throwing more AI at the problem doesn't always converge.
There are many reasons to feel optimistic about AI, and ultimately its general ability to help science and mathematics.
I see no reason to feel optimistic about the future of mathematics and AI based on the current path of frontier labs, unless the misalignment Tao is writing about can be reconciled.
How can we possibly know this when we haven't even seriously started on the endeavor of actively reverse engineering these AI-generated proofs? That's a proper job for human mathematicians, because the AIs themselves are demonstrably clueless about what steps in a proof are genuinely interesting and load-bearing from a human POV. This is evidence of a limitation in AIs' capabilities, not of any kind of misaligned behavior. The fact that Tao actually uses that term in his complaint is deeply disappointing.
Algorithmic verification is a very unsatisfying answer to the problem (e.g., surely it's not just dumb luck that every single case happen to have this exact property), but that's an entirely different issue than saying that no one follows logic of the proof method itself.
Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?
It's prescient.
Could AI write programs that humans can’t understand or debug? Probably, but that’s not what Kernighan was describing.
Can you give an example of this?
Imagine a world where these most complex mathematical problems are not accessible to a few hundred people, but a few hundred thousands people. ...Those original few hundred gifted mathematicians would have an even more prominent role, and their names and achievements would be known by orders of magnitude more people that they are now.
Based on current reward models, the frontier AI labs will burn down mathematics as an impressive display of capabilities and in doing so, will make it impossible for people that get paid to do mathematics to stay employed.
If your job is literally to publish papers, and OpenAI and Anthropic decide that making an infinite-paper-printing machine is the best thing to show how effective their tech is, then as a demo, they destroy that industry.
I don't like nuance here. I think progress is really measured by what humans are able to do and understand, not machines. It is significant if we find problems we struggle to solve. That tells us something. What does it take for humans to solve these problems is related.
The best analogy I can give is if you wanted to climb Mt. Everest you might ask someone for guidance. Would it be better to ask someone who has climbed Mt. Everest or someone who took a helicopter ride up near the top and then went to the peak? This is like the AI versus human gap to me. The helicopter is like using AI to generate a proof. The person who actually climbed Mt. Everest has firsthand knowledge of the experience. Same thing for a difficult proof. The struggle people have is actually valuable here. Likewise, we know people are actually capable of climbing Mt. Everest but if they had only ever rode a helicopter to the top, the knowledge of climbing it would not exist, and surely that is meaningful knowledge given the risks.
So if we rely on AI for proofs I think we lose a sense of what is difficult and why. We lose a sense of what human achievement is. Surely climbing Mt. Everest means more than taking a helicopter up? For students, why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now? This would have the affect of destroying knowledge.
(please do not nitpick the analogy because it's the best but perhaps a clumsy way to describe my thoughts)
Depends on if I want to go by helicopter myself.
> From my memory I think they said it took 88 hours to solve a Millenium Problem versus the decades of time humans have put into it.
Keep in mind those ~88 hours were spread across ~10,000 simultaneous agent instances.
So, roughly 880,000 hours of compute.
Assuming a fifty-year career, and forty-hour workweeks, a human mathematician's career is about 100,000 hours of "compute".
I suspect that with six good mathematicians spending their whole careers primarily focused on it, and working together closely, Navier-Stokes might well have fallen already.
The perverse incentives of academia mean this has never occurred.
The perverse incentives of industry mean OpenAI intentionally scooped researchers who were getting close (granted, with AI help).
I'm not trying to dismiss the achievement - if the proof turns out to be solid, it's quite impressive (though much less so if the training data included the recent human breakthrough, which seems pretty plausible).
I'm just pointing out that "88 hours" is a very misleading way of framing this.
> The perverse incentives of academia mean this has never occurred.
This. Mathematicians in their most energetic years are trying to get tenure or land a tenure-track job. They are disincentivized to go all-in on ultra high risk, high-reward problems. The potential downside is just too forbidding. It's much safer to develop a research program in a mainstream field that affords many opportunities for partial progress that can translate to a robust publication record.
I'm just trying to point out that economically, we have not yet reached the point where AI mathematics research is a no-brainer hands-down win, no consideration required.
It might already be a win, and certainly the ability to compress those 900,000 hours of effort into an actual week of linear time is mind-boggling and potentially a huge game-changer for all kinds of open research questions.
It's not obvious that human math research is dead yet.
Maybe soon, but not yet.
There were more than six doing that and it's essentially why it was ripe for AI to finish it off. But the finishing off was quicker than anyone expected
What teams of people spent an entire career working together, focused entirely on Navier-Stokes or problems they suspected were related, without any "publish or perish" concerns?
I realize tenure is a thing, but my limited understanding is that a significant amount of time is spent earning it, once you account for Ph.D. program and the years of needed to be granted it.
Building a machine that solves Millennium problems is pretty cool too. You wouldn't know it from reading these stories, though.
> why bother grinding through all the material of climbing Mt. Everest and then attempting it if the helicopter ride is how things are done now?
I think you answered this yourself earlier:
> I think progress is really measured by what humans are able to do and understand
People want to make this progress. Therefore people will "grind Everest" as a mathematical community, and that is maybe not so hugely different from a lot of previous mathematical work.
There's still ample room for creativity: simplifying, generalizing, asking new questions humans are interested in, ...
Otherwise we are truly lost.
The current wave of AI slop mathematics might end up driving the next generation of mathematicians away from the subject for the same reason that Mochizuki would have convinced me to quit if his proof had been accepted by the community. Luckily, my professors had the taste to immediately recognize that it was garbage.
This also sounds like a vector for trolling the community with complex putative proofs hiding a known flaw.
Edit: Yup. A bug report to Lean was disguised as a "Collatz" proof in a humorous way. Links below.
Say that AI gives you a Lean proof and says it proves Theorem X. It could just as easily give you the same proof but claim that it proves (not X). How would you know the difference?
Nothing can really be considered proven unless a human expert can read the Lean proof and determine that (X as defined in the Lean proof) corresponds to X. The proof (at least the statement of the theorem) must be intelligible to humans to have value.
It's possible people will just start taking AI at its word. Maybe AI says "Here is a Lean proof of X" and we all just shrug and go "Okay, X is proven." But that's not how it works right now for human mathematicians. Why would we apply that standard for AI?
The statement of the theorem has to be correctly translated from English into Lean code.
It’s like translating user requirements into code. The code could run without bugs but not do what the users want.
The only way to know the AI did it correctly is to check. You can’t just take it at face value.
The "magic" of lean is that (in principle, assuming lean is sound and the proof is verified) that is all you have to check by hand. That is a big deal.
I think a better comparison is: mathematics just becomes like mining bitcoins.
In reality, the value of Bitcoin is determined by humans (even if indirectly, not by planning), and I think the OP’s point may have been that maths proofs can be regarded similarly. No intrinsic value, just what humans find in it.
I'm not a fan of knocking down things that work, however I also find it hard to be against death of the gatekeeping old guard of any industry.
I think math is just gonna have to suck it up like every other industry now. Math productivity is longer out of reach of the average grad student. Like every other industry they are no longer untouchable and are gonna have to adjust to the new way of things or market forces will do what they always do which is refuse to fund ineffectiveness.
I've had to accept that tech/IT will never be the same. Just how it is. You can thrash against it all you want.
Why would research be closed in one direction? Even if AI or human says "Tried that, didn't work" or whatever, someone (or something I suppose) might very well retry it in the future, if nothing else to reproduce it didn't work, in theory at least.
But to many commentors he's a now gatekeeping AI-hating Luddite clinging to a dying profession out of bitterness and envy because his position is more nuanced than "throw AI at everything and turn off your brain".
That is probably the wildest misinterpretation of sth I have ever read here. Misalignment is used as in the goals and interests of ai COMPANIES are not the same as the ones of the mathematical community. It is very proper use of the word, and if anything imo the properest, as it refers to actual people and institutions to which actual self-ascribed goals can be defined, as in contrast to hypothetical superintelligence. AI doomers do not own some trademark on the word "alignment".
The word is being used entirely correctly, and I'm sure the nod to the AI usage is quite deliberate.
This is exactly the assumption that Tao is smuggling in with "misalignment" talk and then refusing to elaborate on any further. Is the issue that AI companies are willfully refusing to provide mathematical insight that they could provide (because they have diverging underlying "goals" to those of human mathematicians) or are they merely working under a capability gap, where current AIs can awkwardly settle major open questions but are not smart enough to provide the kind of understanding and insight that the community of human mathematicians relies on? These are two very different problems and by foregrounding the word "misalignment" in his letter so openly (as opposed to talking about AI capability to provide valued insight), Tao is picking the more adversarial reading with zero proof or motivation.
Again, nobody is accusing anyone of deliberately trying to harm mathematics. It's about misaligned objectives. Eg. Tao wrote the following before the Navier-Stokes announcement (referring to exactly the project OpenAI was undertaking):
> At this point, I would not be surprised if one could batter out such an extension by pouring an enormous amount of compute and AI assistance at such a task. But such an exercise does not particularly hold my interest; I am far more interested in digesting the proof methods and extracting out the key new insights uncovered by this approach.
And then went further to say that such activity could be actively harmful to the field. (All this can be read on Mathstodon: https://mathstodon.xyz/@tao).
The misalignment is that this activity that he and 24 other Fields medalists think is actively harmful to their field is deemed by OpenAI and others to be worth ploughing vast financial, human and compute resources into.
That's the far more sensible reading, so thanks for confirming I guess. But then the misalignment talk is pretty clearly a distraction.
> ...And then went further to say that such activity could be actively harmful to the field.
If true (and there is as of yet insufficient evidence of this), that's merely a contingent fact about very real institutional misalignment within the human mathematical community, not about AI itself or even AI frontier labs. There's simply zero inherent reason why providing a bare truth value or a completely inscrutable proof about the status of some open conjecture should make that entire subfield of math "contaminated" for the foreseeable future when it comes to extracting further human-relevant insight. That's the misalignment we should be caring about.
AI can be used to advance/deepen understanding, or it can be used to superficially go settle a whole bunch of open problems in a field without helping really in understanding them. It all depends on who uses the AI and why. Essentially, it is exactly the same concept as using the AI as a course tutor vs having it do your homework. Or using the AI to write millions of lines of code that nobody can actually read, vs keeping overview of what is going on.
In math it is probably worse because there is no objective function to maximise. Some people here think that the objective function of mathematics is to prove things, which is actually wrong. Mathematicians are not only maximising an objective function, they are also defining the objective function they need to maximise (they are defining which problems to study). The problem with AI/ML is that it can be pretty good when the goal is to maximise a set objective function, but not to set intentions and goals themselves. I would not call that a "capability gap" because we can actually get to have very useful and smart AI systems without ever reaching that point.
Though it doesn't take even an outsider to question the matching here, just look what the only guy who settled a millennium prize what he thought about the community. Or ask some actual PhD, postdoc, or even better someone who dropped along the way, how does it feel to go through this community. Not sure the goals exposed in this article are really helped with the community as it is. One again, this doesn't mean everywhere encompasses the same issues and everyone is acting badly on is own.
But if we want to accept happy pink shiny depiction of a community, we should be giving the same generosity to other communities à priori. All the more as AI and mathematical community do share a large set of common individuals.
The entire point of the paperclip maximiser is the AI isn't evil. It isn't trying to hurt humans. It just doesn't care about us.
Nobody claims OpenAI and Anthropic are out to torpedo mathematics. Just that relative to their internal goals of getting publicity ahead of IPOs by winning awards, what happens to mathematics and mathemeticians in the long run isn't a real concern.
The Holocene [1] features relatively few species humans set out to eradicate. We mostly realised something had gone extinct after we accidentally destroyed them. That is what originally misalignment meant in respect of AI.
- a rare low likelihood sequence of coincidences
- deliberate frontier lab behavior, setting intellectual interrupts on LLM "aha"-moments, so that when some mathematician explains for the umpteenth time the approach they want to take "stop paraphrasing my approach, start the actual calculation!" and when the chatbot eventually groks it, they can scoop in, possibly with live dash-board and interrupt priority levels ranging from "ensure this sample gets into the new dataset" to "Millenium Prize Scoop Opportunity Imminent, call Sam"...
There is always an alignment problem, incentives to ideals, behavior to incentives, ...
And yet as a society we typically consider these technologies a net positive these days, because alongside all the political instability and violence that followed them at first, some people were figuring out how to do beneficial things with them along the way that did more good than bad. Because the issue really was that the influx of more voices and ideas shifted the power dynamic, required relearning how to communicate, and in the end be better as a society.
The issue was figuring out how to "domesticate" these wild new communication channels. One successful example of which was the invention of scientific journals and papers.
Shirky predicted the internet would probably lead to a few decades of political instability too (about fifty years was his guess), and we definitely seem to be in the middle of that process.
Now, I haven't checked his stance on LLMs. I also don't know if I would quite call them a medium for mass communication like the others (as used today, they take humans out of the loop rather than let more voices join the public discourse), but I feel like they are similar enough to otherwise fit the pattern.
Tao's stance similarly feels about wanting to domesticate this wild animal before we get mauled by it. And to stick with the metaphor, I feel that most of the time the AI industry is trying to bamboozle us with spectacular rodeo displays instead.
My point is just that I feel a little optimistic that the human culture around math ("ideas we disseminate in talks, private discussions and careful writeups, connecting them to the previous ideas of others" and so on, to quote the declaration) is robust even against relatively irresponsible use of AI (i.e., an onslaught of proof-slop).
And I guess we'd better try to be optimistic, because even if the declaration results in some realignment between the community and big AI companies, the models capable of this work are not always going to be exclusively under the control of those aligned parties.
That said, I think the declaration is great and I support it--let's see what comes out of it.
Yes, the trailing edge will catch up quickly.
I wonder how long until P vs NP falls.
The online comments I’ve read that side with the letter explain that the problem is AI proofs are inscrutable and useless, but the reality is using computers to brute-force things has long been part of proofs in one way or another, and AI proofs range in legibility up to 100% (like pointing out a proof exists in a long-lost paper, or writing something that is basically correct and just needs a human reviewer to fix it up). People are probably defending the declaration inaccurately, but I think that’s because the real argument or arguments are unclear.
Is it that students won’t learn math if there’s ChatGPT? That could be discussed.
To a non-mathematician (MIT physics and CS ‘06), the letter sounds like, “We have a fun job. Sometimes there are no practical applications of the work, so it’s just kind of like a sport, er I mean science. If someone solves a problem that has stumped mathematicians for decades or centuries, we worship them as a great mathematician. It’s a status thing. So we don’t like some guy with a computer coming along and solving our problems. The way things unfold with all the ideas coming from humans, over time, maybe it’s slower, but it’s nice.”
I don’t think anyone can really stop someone from using a computer capable of solving unsolved problems to solve unsolved problems, and there are always going to be mathematicians who DO think it’s fun to try to figure out what a computer is doing (if the computer isn’t already explaining it in English, which it generally can), and make progress that way, and it obviously will lead to mathematical advances in human understanding, from my point of view. So the whole thing is a non-issue that no one can do anything about anyway.
That's said, I'm a non-mathematician, but a computer scientist. Mathematics was always my weak part, because solving problems which doesn't mean anything doesn't motivate me, and when I'm not motivated, I fail.
Without digressing so much, I want to say that, some of the things in mathematics and mathematics adjacent sciences baffle me. Many mathematicians don't understand the proofs of others, yet they accept it since it checks out within the rules of mathematics. Moreover, many engineers don't understand the formulae they work with. I have developed a very performant Boundary Element Method evaluator, yet I don't know the reason of taking Gaussian Integrals over the surface. The only answer I got is "because the method works that way".
So, if computers can make the proofs now, and we have no curious students pecking their professors to understand how these works, the whole mathematics as a science could wither and die. Because besides it being a sport, it's the very language which can model and explain anything, but it's very hard to master and understand due to that nature.
Our professors warned us against using too much Mathematica, to make us learn things the hard way and make the knowledge permanent. Now, if we offload science to a matmul engine, it's possible that we lose the connection and never able to close the gap in some cases.
Again, it boils down to "This machine has no brain, use yours (or lose it)".
So let's just quickly agree that the actual quote is “However, the push by AI companies to solve mathematical problems as a benchmark is detrimental to the science of mathematics, and to the mathematical community.” And that in this, "as a benchmark" is load-bearing.
We wouldn't see nearly the same amount of contempt from researchers had OpenAI picked a research-friendly approach.
What they did: Hear a rumour about the problem being solved by other researchers, then rush to scoop them (unethical), then, when they actually go talk to them, they try to oust an author (also unethical), and when they finally decide to share their own work, do so in the least useful way possible.
What they could have done: Upon hearing the rumours, connect with the researcher in question and propose that they join efforts instead; set up a joint project to test if the machines are useful in any way, and if that's not appreciated, back down again. And instead of dumping only an undigested paper* and a Lean proof, do the digestion prior to publishing anything (as Buckmaster was in the process of doing). If their own lack of competences was keeping them from digesting it, then again, reach out to the researchers to understand if anyone would be willing to do so.
In the second of those two worlds, we wouldn't be seeing nearly the amount of outrage that we are seeing right now.
*: Here, “digestion” is the process of turning an AI slop paper into something humans can read. LLMs can indeed sometimes (if much more rarely than marketing material from the large LLM companies will suggest) produce correct proofs, but they are often written in bizarre ways – they'll use lingo that doesn't exist, seem overly pretentious, dwell on extremely easy steps while glossing over the hard ones. Currently, a real researcher will take that output and transform it into something that others can understand, use, and build upon. This is not so different from what happens when using it to write software, although as someone who does both, I will say that the amount of digestion needed for proofs tends to be orders of magnitudes larger than for code. This meme is quite accurate: https://mathstodon.xyz/@tao/117068266071803252
The pessimistic scenario is the AI labs will continuously hoover up new developments in mathematics and gazump everyone in their respective fields. It's clear they have no desire to participate in any silly academic niceties, like properly assigning credit or expository work for normal humans. What incentive then do humans have to do this work ?
Generally the mood amongst research mathematicians is pretty dire, and I don't really blame them.
To be clear I think AI is super useful for mathematical research, the problem is the methods of the big labs are massively disincentivising mathematicians from engaging in research, and other associated tasks like giving seminar, teaching writing books etc. These arguably have much more value than finding an obscure counter example to Navier-Stokes.
I'm afraid of the ripple effect of the agenda pushed by AI companies will have. In future and even now, they say AI has significantly progressed math and scientific research in general. There is truth to this, but the narrative has done more damage (so far) to the students, researchers, and the culture of knowledge transfer in academia. Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
I guess, only time will whether this is for the good or bad. And how good AI models get without new data from research and experiments.
Just compare with the comments on https://news.ycombinator.com/item?id=49639408
To use a metaanalogy from chess (once again), mathematicians play the opening game, and builders play the end game. AI is sort of a middleman connecting human understanding to applications.
I think there's a Technical argument to be made that openAI is a threat to the game itself. For example, could it have produced the navier-stokes counterexample without human inputs? since it seemed to have used the much gossiped research strategy "C" and "D", you can't absolutely be certain that Son of Astra (son of altman?) was magicking an unknown unknown from nothing (sorry to cue Rumsfeld). You have got to wait for the other five problems to be solved after general boycott
Subpar PR engine of the OpenAI leadership might kill the pipeline of inputs that they won't admit they still need in this dreamtime before "recursive self-improvement". You can call that emotional. Personally I would rather accuse mathematicians of "preferring local models that believe in the usefulness of unidentifiable individual contributors, and the uselessness of named generalist managers (ie the prompt writers at oAI)"
Big man tlb likes to say that science might be dead but engineering is just getting started. Navier-Stokes is the hammer of the nail in the science coffin. It kills science by killing the prestige of science. The engineers have to imagine that it's likely they will now get all their design ideas from the hypothetical future datacenters.
At the current rate of advancement that phase will last, what, all of 6 months if we're lucky?
On why some people are making emotionally charged claims, my guess: This affects the core belief of what is right or wrong, Impressions based on past doings of OpenAI, losing trust for OpenAI based on sequence of events.
I don't think we will get to see any clear evidence. I'm not even sure what would be the evidence. I would be surprised if OpenAI comes out clean if they have made a mistake. They move on to the next shiny thing.
“For 19.99 a month you too can be a world class mathematician!” Meanwhile they are extinct.
I mean, we have a Millenium problem that is most likely solved. Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could.
No. We should credit that work is being done that otherwise may not be. I'm not defending OpenAI. I'm defending AI in general, something both OpenAI and Buckmaster (and his co-author) used.
No, I did not. If someone says either X or Y happened, they didn't say X happened.
My exact statement: "Either by OpenAI on its own, OpenAI stealing another's work and finishing it or a researcher using AI to iterate more rapidly than a human alone could" [1]. (Either X, Y [a and b] or Z. An Oxford comma could have helped.)
At worst what OpenAI did was inconsiderate, but not referencing prior work is a minor academic sin. This is especially the case when AI is not playing the vanity game that seems to so inflame some mathematicians.
We could also acknowledge that everything that could justifiably be considered stealing is not necessarily covered by current statute.
You seem to be a little sensitive about the subject. Which AI firm do you work for?
I don't work for any AI company.
There is strong evidence that Iran was not as close to a nuclear weapon as you say.
We will find out the truth eventually, like we found out that the WMDs in Iraq claim was a complete lie.
Thank you for commenting this. I'm tired of hearing about Dune like its some kind of utopia
The way Jihad presented in the oblique references in books, anyway, it sounds a great deal like a fanatical fundamentalist religious movement to me. Those are not known for their fine reasoning.
(Coincidentally, this is also the main threat in the original books, just manifesting in a different form, with prescient individuals "locking in" the rest of humanity.)
Wait wait wait. The research mathematicians brought this AI dystopia on us. Not the techbros, not Silicon Valley, not the global financial system and its tentacles, not surveillance capitalism... No, it was the postdocs working on ultrafilters for C(2,3) epimorphisms on symplectic manifolds. That's certainly a take.
Those who think it's me or the machine will fail.
Those who realize how much you can accelerate your research with the help of AI will succeed.
If only though, because:
> There will be no curiosity, no enjoyment of the process of life
How does the ability or inability of AI to do something blocks your curiosity or enjoyment of the process of life?
Not being able to pay bills because you're out of job does ruin the enjoyment of life, sure. But the AI is not the problem there; the economic system is. Torches and pitchforks should be properly applied to the economic and the political elites, not to data centers.
These two things are surely related eh?
> Torches and pitchforks should be properly applied to the economic and the political elites, not to data centers.
What's the difference between these? You can't actually separate them meaningfully, unless you, uh, nationalize the data centers at which point... :|
Not really, no? If anything AI is the tech that is the most likely to disrupt this economic system beyond the point of recovery.
> What's the difference between these?
The difference is that one is a piece of hardware, and another is the person deciding what that hardware will do, and to what ends.
> unless you, uh, nationalize the data centers at which point...
The data centers are not the problem. The fact that you need to be employed to have any kind of decent living at the same time when automation is getting good enough to displace people en masse across most industries is the problem.
You don't even need to go full socialist to solve that! Just tax AI providers on their revenue proportional to the number of jobs displaced by it, and use those funds as a pension for the displaced workers.
It didn’t lead to a lot of people having a wildly successful career, it lead to a lot of people getting burnt out, exploited, underpaid and generally disillusioned.
There will be a lucky few, who have the benefit of being given the space to work alongside. The vast majority of people will (unless we change things) simply be made to take whatever the machine outputs and call it a day.
If you set a wrong foot and start trusting the model outputs, you can waste years searching for nothing.
How can someone realize this? By getting proper research training, failing, and learning from mistakes. For people beginning their research, it would be really hard to make decisions to move forward.
It biases maths and theoretical physics towards the rich.
That one thing that was free.
Yes, but, in the last week we saw an AI lab front-run[1] the research of mathematicians doing what you suggest. The lab threw something like $15M of compute at a problem and the researchers were able to spend nowhere near that. I think the authors are more concerned about that kind of asymmetry and race to publish the results.
[1] - I am not going to debate whether that was deliberate on the part of the lab or if it crept into training data, etc. I don't know and don't think it matters towards the point of the authors here.
> Those who realize how much you can accelerate your research with the help of AI will succeed.
This is only true up until a point. If I treat a mid-sized model (say, Qwen3.8 Flash Next) like a pair programmer, then yes, it accelerates my work.
But I can already see the next stage with Fable: If I give it a couple of paragraphs of spec and $50, then I can just leave the room and go wash the dishes. I learn nothing, I participate in nothing, and I bring nothing to the process. I am no longer succeeding at all. Fable's succeeding without me.
Now, in this model generation, Fable starts getting sloppy after a few thousand lines. I can still build better at scale.
But I don't expect AI to accelerate humans or improve our productivity for long. I can already see the first signs of a future where the AI doesn't need us for anything at all.
Software has always progressed this way, lots of devs back then would work on business websites that have been 99% replaced by wordpress, squarespace and instagram.
I'm sure it's the same with research, you're going to tackle problems that would have not been worth the effort or outright impossible without AI. The old stuff that you'd work for months, yeah that's going to be a prompt away.
If they are not needed to understand the result, then what's their role? Asking the right questions? But how will they know what questions to ask if they don't have a deep understanding of the domain earned by sweating the details themselves?
And how long will they be needed to ask the right questions, how long until AI can do that too?
> Software has always progressed this way
These platforms took on the order of a decade to mature, during which people had plenty of time to learn what's next. With LLMs we went from one-shotting functions in 2025 to entire projects just over a year later.
What do you think you will be working on in a decade?
You're exaggerating the LLM progress a bit but yeah progress has been crazy. Yet not much has changed right? We mostly have the same jobs, just code wayyyy more than before because things that wouldn't be worth it now are worth it.
10 years from now I have no fucking idea. But I'm sure in the meantime those who leverage AI will do better than those who yell at a cloud.
I can't help but be pessimistic about AI moreso than any other technology. Why? I used to be excited to learn the next big thing because I knew it would unlock much more things to do and get paid for, and there would always be more for me. With AI, the new things come nearly too fast, are not very deep or satisfying, and it's hard to see that there will always be a place for me. And so while I leverage AI in my day-to-day, I will yell at the cloud, too.
But in the near future and on a personal level I think we'll need to adapt and pivot but we'll manage.
LlMs don’t understand, they generate, though they have fooled a lot of people who should know better.
The right questions are very simple to ask.
How do I get food. How to cure aging. How to turn lead into gold. How to fly high. What is the ultimate theory of physics. Are there any odd perfect numbers. Is there a soul.
We have reached complicated questions requiring deep knowledge because we tried to solve the simpler ones and reached obstacles. For example to solve alchemy we had to develop nuclear physics (and in the process we got chemistry). If one has a genie able to solve questions, you won't have to think about the complicated ones because the genie will.
I am finding it still takes awhile to get my apps to a happy place with AI. Not because the AI is bad but because i have to sit with it for awhile. Figure out what's working, what isn't, what's missing, what turned out to be kind of useless and in the way.
As a different article said, we still need taste.
The people that uses Adobe Photoshop also did not learn anything about brushes. We are just going to operate at a much higher level of abstractions. You have to think about the question, "If I just prompted this solution, why didn't they?". This question can even be asked now, why do I even bother paying someone to do my taxes? To write my webpage? To host my website? To manage my network?
The real question is, why is this a bad thing?
Every task that is automated is a task that humans no longer have to do. It doesn't mean that humans still can't do it for reasons other than "because it needs to be done".
And if the answer is "why bother if X does it better", then what does it say about the motivations of doing it in the first place?
So many people who are excited about AI making them more productive are, I think, drastically overestimating how much value they are adding to that process.
This is a very disrespectful way to make a point about acting with integrity.
You should consider that maybe your views on what makes something ethical or moral are not universal -- and that coming to a discussion with the assumption that your position is the only valid one is not conducive to convincing others who disagree with you.
I now realize many people have different tolerance level for this.
> Many graduate students (I know) are having a crisis if any of their research worth it? If AI can (or will) do everything, what's the point of doing experiments and all? This will eventually deter a whole generation of curious minded students from research.
we stumbled into a way of brute forcing intelligence with gradient descent.
You must be new here.
I genuinely enjoy doing this, it's really fun to think critically about what an author wrote or how a particular approach works.
But it does make you wonder, why bother? Probably a frontier model could one shot my algorithm in a day or less. It's incredibly depressing. At least I'm not forced to use it now, but I fear I will have no choice after I join academia or industry in the future.
The greatest predictor of AI productivity I have seen has been operator maturity.
AI usage tends to atrophy skills, unless used in a very mature and self aware manner.
All of which depend entirely on the operator having developed a mature theory of mind, and experience with living and working within their own head.
Job market worries aside (though relevant), humans understanding complicated stuff is something beautiful.
Are you sure they’d one-shot your algorithm? Maybe they’d come up with worse, maybe better, but you are in a perfect position to actually judge the validity of any claims a LLM makes in this realm, which most people are not.
As to why bother; our knowledge will keep expanding even with new tools, don’t believe either the hype or the predictions of doom.
One reason not to try is to create an air gap between the OP's ideas and the data that future models can train on. We saw that the mathematicians who trusted OpenAI and Anthropic with their preliminary work found the rug pulled out of their feet by those same companies.
Another reason is to avoid inadvertently plagiarising the work of other mathematicians. Any mathematical insight that comes out of an LLM is the result of training on the entire bibliography of mathematical research, but those insights are spat out of the models without attribution. If you use AI in your matematical research you are only using the work of others without even knowing who they are and what they contributed.
And yet another reason is to avoid polluting your mind with the ideas that come out of the AI. Maybe you get a hint that pushes you to one direction, when you would go into an entirely other direction without that hint. And then maybe that becomes a habit and you can't find new directions without asking the all-knowing oracle.
tl;dr: opsec, integrity and independence are the reasons to not use LLMs in your research. I don't.
Plagiarism is an interesting point, though honestly I think it would be fairly easy to work out who had published similar research if the LLM gives you an idea - personally I see this as the weakest argument against using them, as long as you are strict about attribution - all work like this depends heavily on the research of others - the LLM is just a tool to aid that research IMO.
Opsec is a fair point, and it might be worth avoiding the completely amoral OpenAI at this point for that reason. There are open models though.
Absurd fantasies like solving a Millennium Prize Problem? How is this not prima facie absurd? And if that's come to pass, why should we believe your bar for anything else?
Neither looks good for OpenAI or those who support them.
Yes there was human involvement. But that does ~nothing to change the fact that this is an incredible leap forward, and one that should be viewed with deep concern.
To clarify, they do attribute the original ideas to Cordoba and Martinez-Zoroa, but they don't seem to acknowledge that their result which they say was achieved "with a great deal of help from LLMs" is also derivative of others' work.
Independence is about avoiding making errors because of the influence of error-prone models. I guess I didn't explain it well.
Academics and scientists build on the work of many others and always have - their work is not possible without using other’s work.
Attribution is a problem here but I don’t think a new way to reference many others’ research and combine it in novel ways is the problem or should be rejected on that basis alone.
So I started saying that it only made sense to focus on problems whose solutions would be useful immediately. I even emailed my supervisor about it, arguing that our efforts were “pointless” in the sense that AI-related problems were much more pertinent and had to be prioritised.
My supervisor thought I was bonkers. I still have the email, though. Quoting myself from April 2015:
> By 2030-2040 we will have enough computing power to simulate a human brain neuron by neuron. Once we manage to create a human intelligence we will be one little step away from super intelligence: just set the intelligence to modify itself and see the exponential growth in action. Our human intelligence is bounded by a number of biological factors (e.g. size of a skull) and even the smart human who has ever lived will appear to be a primitive ant to a supper intelligence (machine intelligence will also have perfect motivation). There is plenty of literature on this if you are interested in discussing this further. > > What does it have to do with research in pure maths? I can say that research in pure maths which won't come handy in the next 60 years is just wasted effort. The super intelligence will be able to do maths way better than humans. I believe a lot of current efforts should go into researching of artificial intelligence (or areas to do with AI) instead rather than the pure maths. I want to be proven wrong but most mathematicians I interact with are too narrow-minded to counter me and they just laugh about even contemplating the above. Frankly I am myself so perplexed that I take the above seriously, but I do and it's hurting my motivation.
I'm quite curious what my supervisor thinks of that email now.
I later realised that my motivation depended heavily on believing I was making a contribution that would otherwise go unmade. The prospect of AI doing that work undermined my motivation, but it needn’t undermine hers. I was trying to explain why I was struggling to continue, though I can see how calling the work “pointless” came across as dismissive.
And then to get retorts of "it's just just rich techies that want to find 'meaning' in their jobs while millions starve in the third world". Why would we expect the most wealth concentrating technology in history to lead to mass benefits for those in the 3rd world?
Second - every time we learned to transmit better from past experience we saw massive expansion in culture and economy, not shrinking. Writing, printing, computers, internet and AI are just gradations on the scale of transmission from past experience. Everything stands on this transmitted experience.
My analogy to the body - every cell has the same DNA (every human has the same AIs, culture and tools), but they express it differently depending on context. And no one cell is too fat, the organism distributes work and energy. AI won't make anyone too rich. Its wealth is distributed in the system.
Why? Is suffering the purpose? What is the goal?
Really? That's what you took from my post? Not enough eye rolls in me for that one.
Pointing out that regressive Ai politics likely are not going to help researchers crisis of purpose is not unethical.
You also assume in your comment that we need new large generations of researchers, and that knowledge transfer through academic institutions is important.
I get the fear. But entertaining ideas of impact to institutions and potentially new post ai institutions is not unethical.
But, here many people are okay with the concept of stealing other ideas, plagiarism, unethical aspects of collaboration in research. At least for me and many people I know, this is no okay. This is the part for morality and ethics. I see that no everyone agrees with this.
The issue is, these AI ecosystems don't effect everyone equally. Not everyone knows how to judge AI model output accurately. There is already a large divide on AI usage in research. The fear comes from this. If we wait longer, it might be alright too late. I could be also wrong with these speculations but better to cautious.
Academia has it's own sets of big problems. Before this AI boom also, most people graduating never stayed in academia. The way graduate schools are structured, I would be happy if few people joined. That would also mean we lose good researchers in a long run.
Otherwise you just sound like a jaded bitter washup.
I do see how this is a problem in terms of assigning credit, but I think the cat is already out of the bag in terms of these models being capable. Even without AI labs spending millions of dollars to solve millennium prize problems, there are plenty of other people who will use them to pick low hanging fruit. I don't think any social solution is going to make things go back to the way they were, where you could share your progress towards a famous open problem without risking someone "scooping" you within a couple of days.
I think that the most likely outcomes are either mathematics becomes more secretive, or there is a more deliberative approach to assigning credit than who was "first" to solve some problem. In the former case, this may slow down progress, and in the latter case, this could mean that credit would become more subjective, and be a continual source of controversy.
Moreover in the past, discussion and idea sharing would happen naturally to overcome the friction of the process. But now when OpenAI is stuck on a particular part of NS for example, they can just throw more capital & tokens at the problem.
I think it's OK that there are some materials designed for a specialist audience, and some materials designed for a wider audience. Technical density serves a real purpose in the former (I'd like to just say "stack" without including an explanation 10 times longer than the rest of my paper about what a stack is and why we are talking about them), and some people expend a lot of effort on the latter (think lectures, lecture notes, textbooks, seminars, blogs, etc--totally appropriate to dig into the motivations here).
It's just the fact that it's a really vertical field, not some cultural failing, that results in pretty opaque stuff sometimes.
But now let chatGPT write lengthy emails unrestricted and now no one wants to read your slop anymore. That's what is being advocated against.
It's more about bypassing the culture and processes mathematicians have developed that lead to human understanding, generating new ideas, and bringing up new generations of mathematicians. (See also his article about "non-renewable mining" of good problems.)
Reducing mathematics to "let's just generate results through an isolated and automated system" is a misalignment since it bypasses those processes.
That it makes life more ends and less means.
Suppose that tomorrow we learn that AI just exploited a bug in Lean and the proof is, in fact, bullshit. Or suppose it is the case, but we never learn that.
Where are "ends" and where are "means" here?
Should the proof turn out to be bullshit, then that system will be revealed to be unreliable. Maybe.
Luddites complained that the trajectory of technology was to allow less skilled workers to mass produce goods via machines owned by factory owners, as opposed to helping skilled workers build up and use their skills while passing them on.
Now we have a lot of money and time focused on LLMs owned by a few companies, making it easier for them to monetize low skill labour(prompting versus art/research/artisanry)
And their complaint was valid from their perspective, but the end result of that process is that now you can have functional and pretty clothing for pennies.
It's true that a lion's share of the gain was pocketed by people who owned the mechanized looms. Now imagine how much more of an advantage it would have been without those people.
Same thing here. We can - and should - get rid of all the megacorp leeches. But the benefits of cheap automation remain regardless, and it's still the same equation - removing the gatekeeping harms those who relied on it to keep the products of their labor priced high, while benefitting everyone else.
To be honest, I feel like the difficulty of reading AI proofs is due to the fact that we are on the verge of being beyond human comprehension. This is a demonstrable fact as no human has figured this out despite the problem being open for almost 100 years.
I can see where that's coming from, but I really don't think it's the case. Even with Astra, the proofs you get are just off in a way that doesn't signal superhuman comprehension. As 9question1 says, a common theme is that they dwell on insignificant steps. Another one is that they'll often be full of terminology that either doesn't exist, or has this weird quality where it looks like it is trying to make some minor insight seem much greater than it is. At first glance, that'll often make it look like it knows more than you, but when it's really just doing the same thing but in a more complicated and worse fashion, that to me isn't a signal of comprehension at all. The bizarre thing is that despite all the "stochastic parrot" style nonsense you'll get in individual proof steps, they still often combine to something valid.
In either case, what all of this means is that the working mathematician still needs to go through, and generally completely rewrite, any proof output by an LLM. Otherwise you are passing the burden of unreadability onto the reader.
It's definitely quite curious that the AI labs are able to push these results through seemingly with pure brute force. Perhaps it's largely a function of how many monkeys you have attempting various constructions on top of the known results and strategies the models have memorized.
That's not true. Alpoge and Buckmaster's related LLM-assisted blowup result (https://news.ycombinator.com/item?id=49605915) utilized a strategy developed recently by Cordoba and Martinez-Zoroa.
Just like how they write software, then :-)
It matters if a human came up with it because of everything mentioned in the article... A mathematician's solution is necessarily built on other's ideas that have been disseminated, internalized, pressure tested etc. Methodologies differ too. AI can abuse its compute resources and generate a true/false or counterexample statements, without laying the foundation that a decade of globalized research would have.
No you can't lol, they're multi million lines of Lean, which is already an obscure language to understand. It's an assault on your senses.
https://cdn.openai.com/pdf/32d9f210-8b73-45e0-91bc-82a30aef8...
I don't think "intellectual poisoning" is really the mechanism that harms the mathematics community.
The harm is if you have a community of mathematicians who are focused on expanding human understanding, then having instant access to a bunch of AI proved results muddies the water about who has contributed what. If someone could scoop any significant theorem at any time by pointing an AI at it, how do you really demonstrate that you have created new understanding? Or that your new understanding is about something important? How do you prove that the AI needed your new concepts to be able to solve it?
What a load of croc. This entire debate is fueled by a perceived lack of attribution. The AI learnt from researchers and did not give them a sporting chance of being first before scooping them. They were expecting some sort of fair play, instead they got a ruthless machine. Every other tangent to this debate is irrelevant, the culture, the community, the shared symbolic growth. Every mathematician I know is secretly trying to one-up their peers.
This goes much broader than mathematics or academia. This is the entire basis via which society distributes its wealth: based on a labour market derived valuation of ‘contribution’.
Correction: that's not how society distributes its wealth, it's how it throws some bones to the masses. I wouldn't be surprised if over half the wealth goes to people who don't sell their labor at all.
Markets defined entirely by law have distorted our collective understanding of what can actually be built with the knowledge our species has accumulated thus far. How will traditional shields that have protected capital accumulation in tech to survive in a world where governments now realize control of technology is a national priority? Especially as we see its impact on modern warfare, and that such conflict looks like it’s only escalating over time.
Mathematicians appear to me (as an outsider) to exist in a field without such distortions, and I think offer engineers a preview of what’s to come. I certainly have completely ceased sharing original ideas online at this point.
it feels like an unintended consequence of the millennium prize is that people view the [last contributor to the solution] as the only one to make progress on the problem. I've never viewed Poincaré as solved by one person and the objective of the prize was to encourage more people to make attempts and contribute towards progress.
this issue is independent, but in these circumstances perhaps interweaved, with the 'ai is taking over math' concerns
AI companies are investing these resources primarily as a marketing exercise. There is no near term commercial value to a 100 page Lean proof of blow up in an extreme special case of Navier Stokes, besides the bragging rights. As the statement says any commercial value in this stuff comes a very long time later after new insights and techniques have been digested, integrated into the mathematical canon, expressed in ways that don't take a lifetime of study to understand, etc. (things that AI is not yet capable of doing itself). The bragging rights, on the other hand, are massively valuable. There is a mystique to maths that makes "our AI solved a Millenium Prize problem" an irresistable headline for a company like OpenAI.
What the mathematicians are saying is stop pouring resources that most mathematicians can only dream of accessing into projects that are actively damaging to their field. They face a massive challenge of figuring out how maths can evolve in the face of this new technology, and this is not helping.
It's this sort of thing that motivates people to burn down the institution you might be trying to defend.
lol, yes, this sort of thing is what many people who voted for Trump were saying, and things are going great for them.
> They prove a millenium result, but it doesn't count because they are bad people.
OpenAI has only themselves to blame for this, and they know it. They could have handled this so much better. I'd bet there's more meeting time right now going into how to unveil future math results than on meeting about the actual math research.
Is it reasonable for any field to make such demands? If this were doctors objecting to AI becoming good at medical practice would you have the same concerns?
While any idea of OpenAI spying on people to pursue their goals is disgusting, the rest of this is par for the course, as Kasparov experienced with IBM in the 90s. Humans still play chess after all.
I doubt OpenAI will take such a combative stance and accuse these mathematicians of "demanding" things, as you do. As I said, the purpose of this is marketing and the statement simultaneously undermines the value of that marketing (showing these projects as irresponsible) and gives these companies an even better piece of marketing in its place: "our AI got so good at maths the mathematicians begged us to stop". It's entirely possible they will stop pouring millions into these projects.
The fact is these fields are supported by society because of the benefits to everyone else. Once the same results can be achieved in a cheaper and faster way that is what will be done. We should mourn this in the same way we do buggy whip manufacturers. Again people still ride horses.
Maybe it's worth double checking that you know how these fields benefit everyone else? Proving the blowup of the Navier Stokes equations in 3D isn't going to make your gas cheaper or make harvesting food easier or make drones easier to protect against. Maybe consider the deeper effects at work?
If you can make breakthroughs on such areas as fluid dynamics, control theory or information theory with AI then that absolutely is a big deal with real technological implications.
Would you expect Fields medalists to cure cancer if you moved them from the math department to a medical research lab? This is precisely the fallacy that the frontier labs are counting on to inflate their valuation as their IPO approaches. They want to use headline-grabbing problems in pure maths to make their models look "smart" in the public eye. But what does "smartness" in mathematics really mean in terms of economic value? It is not at all obvious whether success in abstract mathematics should translate to successes and, more importantly, profitability, in more grounded endeavors.
Look at OpenAI's job postings (https://openai.com/careers/search/). Those roles involve far more pedestrian yet profitable duties than research mathematics. So why isn't OpenAI automating them with their vaunted models? Success in one field, no matter how "difficult", does not predict results in another field.
Last weekend I spun up a small agent swarm and pointed it at a field of math I have some affinity towards. Within four hours I had settled three conjectures, one of which is rather famous (for the field, not in general). It cost me about four hundred dollars.
I am at a loss about what to do with these results. On one hand I feel like the mathematicians working on these should know about them, but on the other I feel a bit like a barbarian who suddenly finds themselves sacking Rome.
They're valid.
The biggest problem is, IMO, drivebys uninterested in actual results, just getting a check mark, and the equivalent of dropping a 200k line PR on people and expecting them to be interested and do the work for you. These are things many on HN are familiar with and know how to do better :)
I can understand why the community is pissed. So now, lean proofs can be churned out at scale, and the community is left to decipher all of that slop into human understanding. There are bad actors with misaligned incentives coming in with drive-by proofs upending what the community holds dear which is to practice and propagate the art. I applaud them for this declaration.
To re-align incentives the following could happen. AI slop lean proofs are dumped unceremoniously into a lean dumpster, and what gets rewarded are results that could digested into human understanding - via the already followed human review process. Prizes are not given to lean proofs since anyone with sufficient compute can churn them out.
If you are trying to understand better the field, then do a good write up of the proofs so that people can learn from it.
If you want to earn the respect of people because you found interesting proofs. Then do a good write up of the proofs so thst people can learn from it.
If you want to plant flags and pollute peoples minds. Then please publish it anonimously, no one wants to correct LLM slop for you.
Probably we should build a repository of AI slop proofs that are only allowed to be publish anonimously. That way people may be more inclined to work on it because they would feel like they are cleaning your house for free.
I like my current life and don't want to get dragged into the current fracas surrounding the use of AI in math.
The problem is with people that may do it without contributing to the community.
The field I've been investigating is not large. Even if I were to take the time and care to beat the interesting results into something meaningful, I'm afraid the pace at which I'm able to produce these results would not be well received.
Mathematics is just the formal system: in that case we will never be able to beat the AI as humans. The goal is then to cover as much of the formal landscape with AI generated proofs to proof as much statements as possible. Success metrics are lines of lean and numbers of proven statements.
Or
The formal system is just a limited representation of what mathematics is: in that case probably some parts of the formal mathematical landscape are more important than others. Not all statements are born equal. And 90% of AI generated proofs will be just noise. Our job as mathematitians is to steer the AI to high mathematical value regions and turn formal proofs into mathematical proofs and insights.
From my point of view we have know since Godel that we are living in the reality of point 2.
Each of the two points implies a very different way of doing mathematics. So pick the one that you think is true and act accordingly.
I would recommend publishing them to Palomar (https://palomar-registry.org/) - I have no affiliation, this is an online registry of Lean-verified proofs created by Terrence Tao.
I have submitted a proof there that's also minorly important in an extremely niche field.
Anyway, I feel like it's a good place to dump AI slop lean proofs because the main point of the registry is that it verifies that: 1) your Lean challenge statement is the same as what you informally state you're trying to prove; 2) your Lean proof actually compiles.
This could be useful to future AI slop researchers who want to know if a given result has already been formalized, and they may be able to mine some lemmas from your work. Also, it's good to know for the field in general what has been proven.
I'm fairly certain you can set your publishing name to be whatever you want, so you could set it to be just the word "Anonymous", or the name of the model you used.
You asked for a painting. A robot made the painting. You looked at it and said, "well, I guess it's good. Should I put it online or something? Dunno. Hey Fred, what do you think of this?"
Meanwhile, your next door neighbor spends their entire life developing their understanding of life through art. They "understand" (maybe not in a way they can articulate) art. You go next door, you look at their painting and say, "well I guess it's good." But you also understand that your neighbor is just like you, and maybe you are a painter in another way.
I find it strange that, people can't see that, we don't need to solve hunger and poverty and work balance, and etc, by a round-about make-super-intelligent-AI. We could just solve it. It's pretty obvious how to, as well.
We can all be painters, if we put restrictions on the psychopaths.
And robot farms are being made now, without AI. Just plain and simple normal robotics.
It's also possible that the result is already known and you just weren't aware of it. It's easy for someone outside of a field, or even one steeped in it, to not be aware of certain solutions.
I actually partially disagree with this. What happened to all the excitement about Intelligence Augmentation (IA)? Now it's AI instead of IA. I think there's so much untapped potential for augmenting our intellect with the likes of https://dynamicland.org and https://folk.computer, as well as the work that's been going on in college math education, things like Lean, etc. I think the only reason human math capabilities haven't expanded that much is a failure of our imagination, not our potential.
The point is that these proofs are largely useless without the insights. The value of a proof is largely in the travel, not so much in the destination.
Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing. And instead of them - and nobody - spending millions of dollars to solve the problem, successfully, they want every problem of their academic industry to persist because even though they never solve the problem, they synthesize and solve lots of other problems nobody asked for. And get to boost their egos?
Yeah, stop that. Actual alignment is on the humans themselves, if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades and don't worry about the narcissistic elements that slow their industry down.
I actually found this to be the case with some basic linear algebra notes I was recently doing in Lean (without using mathlib). The model could generate working proofs, but they obscure the basic ideas (actually I wonder somewhat if this is because the Lean code that's out there to train on doesn't make a huge effort to read like textbook proofs, which was my motivation in the first place). I give it a skeleton of a couple lines of `calc`, letting it fill in the reasoning for each line, and it does much better. Then ask it about making some macros to simplify "trivial" or "obvious" things, and it does even better. etc.
I suspect there's a good workflow where a big SOTA model makes an impenetrable proof (or code) and then a human works with a FIM model to simplify it (with the larger gnarly proof right there in context for FIM), but unfortunately everyone seems to only care about agents right now.
Presumably you're a human. Are you going to do that?
Do the job yourself. And if you can't, that's maybe a hint that you should listen to the people who can.
I'd be happy to do the job. Actually I still dabble recreationally (clarifying Codex's Lean proofs, even!). But like I said it's one of the most competitive fields on the planet. As you say, you have to dedicate your life to it.
If a slop proof isn't helpful, they don't have to "toil unwillingly to understand it". They can just proceed with the knowledge that the proposition they want to prove 1. is true and 2. is provable, which is already a decent start for motivation. But often LLMs can actually do quite well explaining ideas too in the hands of an expert. Or you can ask them to prove some technical lemma that you think ought to be true, and that could offer insight for the thing you're really interested in, but for which the details are actually not all that interesting to you. You don't have to one-shot "prove RH from the ground up in 50 million lines of Lean."
To me this analogy points in the complete opposite direction. Imagine somebody takes a half-completed project design you're trying to figure out, vibecodes a rough prototype of it, emails your manager to announce that the project just launched in alpha, and then dumps it back on your lap for approvals and testing and productionization. Would you say that they've added value to this process? Or did they just strip away all the hard parts of the problem so they could claim credit for the easy part?
If that person then runs around telling people that they're the real author of your project, because they generated the original POC, would you consider that an accurate assessment?
But mathematicians define their field. They're smart people. They're capable of recognizing when someone just did a vibecoded throwaway PoC and when someone has a well structured proof. Actually even before LLMs they'd publish new, clearer or more elegant proofs of old results. They can say that inscrutable proofs are exactly as valuable as they are, and that the first explanation people can actually understand carries its own prestige.
I'm also not sure I understand what you're objecting to if we agree that mathematicians define their field. The source link is a declaration from 25 Fields Medallists with precisely that goal. They believe/define/declare that the type of AI-generated proofs we've seen are vibecoded throwaway PoCs; they feel that a well-structured proof must include factors such as "a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others", and the success criterion is not a true/false conclusion but rather "development and integration into the mathematical canon".
My objection is characterizing things like AI slop proofs as not valuable. Obviously it's valuable to know that:
1. NS has solutions that blow up in finite time.
2. This fact is provable, and we have a proof.
I don't think anyone anywhere is saying that these will replace mathematicians as they are now. I also think "a proper write-up, the isolation of new methods and ideas, and citing relevant previous work of others" is frankly not necessary or even desirable before we publish a computer generated result. We have a machine now that can spit out answers that we have good reason to believe are accurate, but they're perhaps inscrutable. There's no need to first decode the why and figure out proper attribution before simply posting the proof online. The proof itself does add value as it stands, even if it's not the ideal. Hoarding it until you can do a proper write-up would be silly.
This letter includes someone like Terrance Tao who publicly expressed a lot of optimism about AI for solving novel math like with the Erdos problems. It's not sour grapes but the first steps to define those new expectations for the future to reduce the perverse incentives.
And yet, predictably, people are accusing him of "gatekeeping" and ignoring the arguments he has made here and elsewhere about the benefits vs. harm in different ways of using AI.
The NS counterxample is actually, by any market measure, a "problem nobody asked for" in the sense that its existence doesn't have any commercial relevance (beyond juicing OpenAI's IPO). So the only long-term value solving it could have is by virtue of whatever reusable theory/insights were generated along the way to the counterexample itself. The letter is absolutely right on that point.
It's not actually clear that those insights will come faster from reverse engineering this LLM proof vs. humans building theory to solve the problem themselves. So what you're saying may or may not even be an efficient way of operating. Also, it implicitly depends on mathematicians to do the hard work of creating problems and then deciphering LLM hieroglyphics for essentially free while the only immediately profitable component gets outsourced to a frontier lab. In what world is that model going to work?
Reading between the lines, it seems like maybe you have a personal grudge for some reason and simply think the technology will advance enough to where we won't need academics at all. But you should say that in the first place.
My stance is that solving the problem is aligned with humankind
the rest is just hypothesizing a way that academics fit in this world at all
Mathematicians agree that "solving the problem is aligned with humankind." They disagree that releasing a counterexample this way actually constitutes "solving the problem" precisely because there is now little incentive to do the hard theory-building work that actually has the track record of leading to human advancement.
I would recommend a bit more humility and trying to better understand why 25 Fields medalists, among them people like Terence Tao (who isn't anti-AI by any means, he's even promoted a registry of AI Lean proofs), are saying this.
> Okay, to elaborate, substantively, their point is that the people using these AI models are not doing it for the love of the game, but for marketing.
No. The point is that AI companies are using the models to solve problems in such a way that the useful part of problem-solving, i.e. the theories and tools developed during the process, is not present. And they are doing that because the companies seem to be motivated not by honest advancement of math but by marketing and publicity.
> they synthesize and solve lots of other problems nobody asked for.
No one asked Fourier to solve series representation of functions when he was studying the heat equation, and yet thanks to that we have Fourier analysis.
> if they want to remain relevant as academics and mathematicians, they need to learn how to replicate the proofs and the steps that alluded humans for decades
The point they are making is that if AI keeps being used as "problem solver" rather than "theory understanding", replicating the proofs and getting the useful parts out of them will be far more difficult.
One of the points the parent makes, along with the TFA, is that academia -- or more specifically, the "mathematical community"-- is a setting primarily for creating and ingesting mathematical knowledge, and disseminating it to the next generation and to other fields. Humans absorb this material slowly, through lots of discussion and collaboration -- it is necessarily a slow process. Facilitating this is one of the important functions of academia. Your usage of academic as a slur here is a bit silly for this exact reason.
I don't claim it is perfect, and we can argue about pedagogy in elementary courses till the cows come home. That's not really material. But this is one of the only settings in which such knowledge is broadly valued for its own sake, and in which there is a semblance of incentive to help others "know" this stuff as well, be they future generations of mathematicians, science and math educators and communicators, practitioners in other fields, or genuinely curious amateurs.
This is the core misunderstanding that the open letter is attempting to correct.
Developing a better understanding of the Navier-Stokes equations could have a number of implications for useful technology. They're fundamental to fluid dynamics, and turbulence in particular is something that many people feel we could work with more effectively if we better understood how and why it's generated. The Navier-Stokes smoothness problem is an interesting and long-standing benchmark for this understanding; we don't know why it should be so hard to answer, so we hoped that the process of developing a proof to the problem would produce more understanding. (We may still be able to extract this understanding after the fact, if OpenAI's proof is fully human-comprehensible.)
Simply knowing that there exists a finite-time blowup is not practically useful. We know that fluids in the real world don't produce random singularities, so the result can't really have much physical meaning. What it illustrates is that the Navier-Stokes equations fail to model physical fluids in some yet to be characterized way.
How much do you think other AI companies would offer to get access to the transcripts of the generation that led to the proof? No doubt OpenAI will include it in their training data somehow and use it to build the next generation.
There is already economic value.
Like the mathematicians working on famous problems in private until they could claim full credit for something interesting wasn't also a marketing exercise for their own careers. The commercial value (or lack thereof) of a proof doesn't depend on whether it was done by a human or a machine.
OpenAI just burned millions of dollars over a weekend after hearing that someone else was close to solving the problems. Their interest was in their AI system more than the actual math problems.
Don’t you see how that’s different?
If clout was the goal I don’t think becoming a lifelong mathematics academic would be the first step
I guess that type of “clout” feels different to me.
Wanting to be validated by peers for your talents in a niche field vs. using millions to try to solve a math problem that you don’t really care about with AI to market the gigantic company you work for.
I can see how what is happening to mathematicians is similar to what is happening to coding.
I’m not sure what your broader point is? Mathematicians shouldn’t be upset? Coders should? Something else?
From your first comment it seemed like you were disagreeing with me but I’m not sure how.
That’s why I asked for clarification.
Edit: BTW I’m a web developer and designer, Not a mathematician
This is an extremely rare situation that practically never happens. Most mathematical research is done in the open, with partial results being published, conferences where approaches are discussed, collaborations... The Navier-Stokes solution is a great example: the approach that OpenAI ended up using was something that two mathematicians had proposed previously and was being studied and followed by several others, with different sub-paths within the same approach.
I can imagine mathematics of the future being more like that rather than history of discoveries with dates and names
When there's a discussion about doing something against the damage of the AI industry: "whoopsy, sorry, another cat escape, nothing can be done".
When there's a concrete mention of an actual solution to avoid more cats escaping: "that won't happen, and even if it did, the damage is already done, and in fact it’s not that bad you all just have to go with the future we decided for you."
So the bag is wide open, more cats will escape, and nothing can be done about any of it. not about the ones that got out, and not about the ones still inside. Sounds more like a preemptive excuse for inaction, cosplayed as pragmatism
It's more than the yardstick for individual contributions. It's also the yardstick for the contribution of ideas. And that kind of yardstick is very important.
Math has a challenge that's only occasionally acknowledged imo, that's it's possible to just go up the ladder of abstraction, formulate ideas but have those ideas actually not be useful or interesting. The ideas that math has developed, that mathematicians consider important, are those that have helped people solve hard problems.
I think you're missing key details in the statement. You can hate problems and sprinting towards solutions, and still have a deep impact in mathematics – Grothendieck once famously said you can often "soak the walnut" to crack open an issue. Rather, it's how frontier labs are inherently commercial incentivized to act like this, because it is in their best fiduciary interest. That is the misalignment.
Baudelaire argued that photography became a haven for failed painters, the sorts of hacks that could not finish proper training. Photography, as a mechanical rendering of the world, could only record what already existed; it couldn't transform reality the way a painting could.
He also criticized the public's craze for "rushing" into it, and complained that this technical "progress" was weakening the arts.
Do you see some parallels as well?
[0] https://fr.wikisource.org/wiki/Curiosit%C3%A9s_esth%C3%A9tiq...
Photography decimated other forms of visual art, so the concern wasn't wrong. But AI threatens the entirety of human intellectual endeavors. I can make do without oil paintings in my home. I'm not sure I want to live in a future where we make do without brains.
In a way, if photography is an ersatz for painting that eventually made imaging available for the masses, then AI could become an ersatz for thinking. But it feels like I'm paraphrasing TFA.
What will happen when AI companies have spent their advertising budget on math problems and whatever else gives the maximum wow effect for the bucks? Probably customers hooked on the vain satisfaction of spending token$ to impress friends.
But it's equally obvious that this has nothing to do with the tech itself.
This is like separating “the use case” of killing people from guns and still trying to have a coherent discussion about them. You simply can’t handwave it away with “well that’s not the tool’s fault.” It’s unproductive and sidesteps the conversation.
And even then we still need them because sometimes you do need to kill someone who's trying to kill you, which is not morally wrong.
The question, again, is how they are used.
My answer to both is the same: nothing stops mathematicians from doing both of those things, with or without the help of AI. And we all understand that it will take time to do that. But complaining about the dawn of a new era of advancements seems counterproductive.
you completely misunderstood the critics. your analogy is awful. this is much closer to the industrial revolution in the uk: it brought a lot of progress, but also extreme inequality and concentration of power.
Are we on the same page?
https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...
> It's not about whether the conjectures themselves are interesting or not, or whether people simply have enough time.
Tao seems to think otherwise, if I am reading him correctly:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions. [1]
> It's that a bare proof made by a machine doesn't actually do much for us.
I get it, and I think the same can be said about all sorts of human endeavors.
> There is not some set of problems that, once finished, will amount to some kind of final, correct system and we can call it a day and, like, utilize it.
Sure. Although there are certainly practical applications to be found along the way. E.g. proving P=NP would be potentially very significant in the real world. I think we agree.
> "Mathematics" is the people doing it (the "mathematical community" Tao references below)
Sure. And the same can be said again about all sort of human endeavors. But I don't see how that is a reason to stop using AI in those fields, either. It doesn't subtract anything, in the same way that chess engines didn't destroy the love of the game for chess.
And just like in chess, these AIs can be used to gain a deeper understanding. Including, but not limited to, explaining to humans the proof they just came up with.
[0] https://mathstodon.xyz/@tao/117237320796901560
[1] https://terrytao.wordpress.com/2026/09/11/a-severe-misalignm...
Also, how, in your words, do you feel like the first quote justifies your point (presumably with regard to the question of "interesting" or not)? And why do you think the second one is more about time itself rather than attribution? Do these things actually contradict the letter above (or the comment on it) in your mind or not?
In general, do you disagree with something here specifically? Or is it kind of a yes/and thing? Does any of this help, in your mind, with the Baudelaire comparison you were at least at one point trying to argue for? Its a bit hard for me to see the argument here, if there is one, just with what you have written. But I am sure I am just not knowledgeable enough to grasp the argument!
Some folks are struggling to adapt to this change. Tao actually sounds like he is doing alright compared to most, even if some of his arguments seem a bit weak, as I alluded to in other comments in this thread.
Hopefully it makes some sense. And if it doesn't, at least we had a nice chat.
The historical record. That's why I am drawing some lose parallels with Badulaire. Incumbents being unhappy about a disruptive technology, lashing against the early adopters, and fearing that it signifies the end of their craft, when in reality it's just a period of change and adaptation. Without the advent of photography we would not have Impressionism nor all the movements through the 20th century. Photography forced painters to reinvent themselves, and LLMs will force mathematicians to do the same.
I thought the historical examples of photography and chess engines would be enough for people to connect the dots, but apparently not.
I guess I am sorry for pushing, I felt like maybe there was going to be something more specific to the point.
I think one thing to keep in mind in the future is that for every Baudelaire, there is a Benjamin or Italian futurist saying the exact opposite! Its not like the fear of photography was a monolithic, shared thing, even at the time. Just, it's not so straightforward to have the "historical record" speak exactly one narrative, and unless you are a full-on technological determinist (is anyone these days?), you can't really conclude anything from the past like this.
You're just an overconfident idiot who thinks that because there is an analogy along one dimension of a absurdly complex problem you've somehow arrived at a superior truth.
I repeat, this is an incredibly complex situation in which no one fully understands what's going on. The letter is simply asking for greater awareness of the implications of what AI companies are doing, they're not even saying that AI is bad!
It's what fanatics do when they want to enforce their view on the world, they have to attack anyone with a reasonable viewpoint because they can't imagine a world where someone tells them they don't like what they're doing.
As for the rest of your comment, I hope you have a great day.
You're locked into that idea, but provide no real argument. The dangers they describe in the letter are real.
It's the work of authoritarians and needs to be rightly called out.
This is what I was alluding to:
I wrote recently about how the collection of good, fruitful open problems is now being mined in a non-renewable fashion, leading to the potential scenario of these problems becoming scarce [0]
It's especially annoying given that many people are making repeat Baudelaire's critique of photography (or Socrates critique of writing) in the context of AI and the declaration is interest because it's not that.
> nothing stops mathematicians from doing both of those things
Actually yes, and they explain that pretty clearly in the letter. These problems have been used as "goals" not because the solutions are necessarily worth it, but because they set a clear direction whose process would probably yield very useful theories, and also more conjectures and problems. If the goals get solved by AI models that do not generate comprehension, then the direction gets less clear, it's harder to collaborate and get funding (think what would be easier, to get funding to try and solve Navier-Stokes or to try and understand an already proven Navier-Stokes?), and we get less open problems to work on.
It all comes down to this. Yes, they will need to give a good and convincing argument to the taxpayer. And at the end of the day, if the circumstances change, and you're unsuccessful in making your case in your plead, the taxpayer and their representatives may not want to keep the same level of funding and instead put that money to use elsewhere. So make a good argument why your job is still worth that cash under the new circumstances. The argument can appeal to aesthetics and art and philosophy and a kind of feeling of religious elegance and so on, it's fine. I can't say how well that will work, but it's worth a try. Taxes also fund theology and other pursuits with no tie to cold hard engineering etc. But that level of funding is not the same as for math currently. That gap may shrink.
- The useful part of math is the tools, the understanding and the theories. Knowing whether the result is true or not is usually not useful. Imagine an AI that just says “cancer can be cured” but doesn’t say how: that’s what the letter is criticizing.
- Terry Tao is one of the biggest proponents of AI use in math. He’s been for quite some time and he shares a lot on his blog.
Seriously, it seems like you’re arguing about some idea you have in your head and you’re not making the effort to actually understand anything about this letter.
These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc. But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
As Feynman said, "mathematicians can research what they please. If you want something else, you work it out yourself" (not verbatim). Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
It was an example, an analogy to argue that answering a question might not be as important as how it arrives to that question. Hence the “imagine”.
> These people are concerned about internal community dynamics like tenure, chairs, PhDs, grad students, the whole ecosystem, papers, hiring, conferences, journals etc.
You just described most of the parts of the mathematical community.
> But the wider population doesn't care about this and academics don't understand that nobody cares about this outside their bubble.
I don’t care about the logistic system and yet I want my food to be in the market. If the population at large wants to use mathematics they will have to find a way to support a mathematical community. Now, if you don’t want mathematics, again, go argue about that.
> Indeed, now people can do this with AI and it makes mathematicians salty that they will have less prestige.
Except that they can’t. I do not know how many more times we need to explain that “answering questions” is not what mathematics is about. That if that path is followed, then mathematics will suffer as a result and the pace of mathematical development will slow down.
Honestly, I think it would be better for you to really stop attributing the arguments of mathematicians that are by no means anti AI to saltiness or other childish feelings. Try to be more humble and try to find what kind of ideas they might hold for them to argue this letter. Of course, that requires reading the letter and trying to understand it.
Just as science (as in the scientific method) isn't the same as Science, the community social practice and logistics and tenure rules and committee compositions and journal page limits etc, mathematics is not the same as the academic mathematics community. They are not the sole producers of mathematics and not the sole users. The funding is largely in the hopes of the usefulness of the resulting math. Not all. Some funding is like art and culture funding, for propagating a cultural legacy, like folk dance also gets funding and experimental performance theater also gets some tax funding. But that's not the bulk currently in math.
I have read the letter. It's all about stuff of holding back because they don't want the answer key because prestige reasons, and how will grad students train their brain if we have too many answers. This doesn't consider that many people, like engineers, use math as a tool. They don't want to hold math back. It's fine to think that applications are too dirty. Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available. They can do the aesthetic math as an art side project like an accountant can paint or sing off the clock. But generally you also don't pay everyone for their hobbies even if those hobbies are nice culturally rich endeavors.
And you have not understood it.
> they don't want the answer key because prestige reasons
No, that’s not the reason. They don’t want the answer key because the answer key is useless.
> This doesn't consider that many people, like engineers, use math as a tool
Of course it considers it. Many people, like engineers, could not care less about the answers to most problems mathematicians work on. They do care about the tools they develop in the process. I’ve already shown examples of this but I’ll do it again: Galois theory was developed when trying to answer whether there are formulas to solve roots of polynomials of degree 5. Fourier analysis was developed when trying to find an analytical solution to the heat equation. Riemann developed his geometry to explore which Euclidean axioms were actually important. None of the direct answers were as important as how they got to them.
> Again, they want funding, they have to explain why exactly they should get it. And it better be an explanation that still holds water with AI available.
The explanation will be exactly the same. Only it will not be just hard to explain why the problem is important, but also it will be harder to explain that no, just the answer by itself is not useful without the understanding. Just like I am here having a really hard time explaining to someone who doesn’t understand how mathematical research works why “just getting the answer” is not a useful output.
> If the AI can digest it better, then academic mathematicians will become humanities professors
Cool, then once that happens we can discuss what to do. In the meantime, AI does not digest proofs properly, does not explain them and does not generate any understanding, and it doesn’t look like that’s going to change. Hence the letter and the criticism made to the approach taken by AI labs. It is not that hard to understand.
I find it interesting how many people are incapable of imagining that the tech capability will not be frozen at today's level and where we were e.g. a year ago and that a similar change may happen until next year. Instead they make sweeping assumptions that the current limitations will be with us for our lifetimes. You need a much stronger way to adapt to the new reality.
It's absurd to assume that because LLMs are improving in certain aspects they will improve in everything, specially when the part they are lacking in is not precisely something that would be a strength of their architecture. I mean, models have improved a lot but they are still not good at strictly following instructions consistently (there's a reason why AI labs are worried about safety alignment). They are still mediocre to bad at software design, even the latest models (haven't tested Astra seriously yet). And it's the same reason they are bad at creating mathematical theories: they do not have mental models like we do, it's not even useful for them. Their comprehension is limited to textual context that they need to refresh and reprocess continuously. That's just how LLMs work.
> I find it interesting how many people are incapable of imagining that the tech capability will not be frozen
I find it interesting that after taking this long to, I assume, finally understanding the point the letter was making, you automatically switch to "oh well AI will do that too".
Baudelaire popped up in this article two days ago.
AI is being treated and pushed as a replacement for every medium.
An LLM can produce a portrait of you as well by generating a facsimile of you based on existing material (or functionally a “guess” if you describe yourself I guess).
In a parallel thread omnicognate correctly pointed out that for AI companies it's a direct commercial loss to pour all this money into bruteforcing the solutions to these problems, and that a lot of times the solutions by themselves are not directly commercially valuable. They are doing it for stock price, trying to lure in private capital in preparation for IPOs.
Their models are good, but they are not the moat because Chinese models are good too, so what they are doing, in my opinion, is more harm than good. Mathematics is a science by humans for humans.
This is tiresome. People should be able to flat-out criticize AI without the implied need to justify themselves all the time or "be careful". Its almost like AI has a trillion-dollar agenda backing it, to the point that you have to add a careful "its really great! But there's this little issue..." for any criticism.
Even those who are very pro AI should have the intellectual honesty of admitting that there are very valid reasons to criticize AI.
The crux of the argument perhaps. It suggests that too many people are currently studying mathematics without making much progress.
It's a capability demo, not the main point. There is plenty of demand for intelligence on tap. These flashy results are necessary though, otherwise people keep claiming AI is just hype, it can't do anything etc. So they need something to point at. And it seems to be working because the anti-AI narrative has changed from the previous stance that it's just the new NFT that will simply go away to how it will replace human intellectual work and will be bad for artists and desk workers.
Realism isn't difficult, so critique shifts towards composition, narrative, context, process, and emotional impact.
Taking a clear photograph is easy with modern equipment, which means the bar for what's considered "good" is high.
There is no doubt that AI has changed the practice of mathematics, just as it has changed the practice of software engineering (and will soon change almost every intellectual job).
Trying to deal with change by saying, "please stop the change" is foolish, IMHO. Mathematicians need to redesign the discipline with AI in mind. But I get that it's easy for me to say that and hard to actually do.
That wasn't the critique. The statements then were that photographs weren't actually art at all. They were the results of specialists with skills but not 'art'. Yet the photograph as a product was displacing real art in the marketplace.
The argument here sounds similar. The fear, as I understand this statement to be saying, is that by being given the correct answer, in the form of a 100-page Lean proof, humans will be robbed of the chance to from insights about the structure of mathematics itself. I don't see any reason that humans can't continue to develop insights as they try to digest the 100-page Lean proof into something more manageable; but with more certainty and fewer false starts.
A but like whenever the first sprinter hits a new world record other runners follow along.
Knowing that something is possible tends to strengthen our ability to work with it.
We will potentially see the same with math.
There is no equivalent in math.
I know nothing about chess yet I dare say that I'd doubt this. Surely chess enthusiasts would be interested in analyzing how a superior chess program came out victorious, no?
Mostly nobody cares about professional chess. The number of people who are actually interested in today's game and not the drama are a tiny sliver of that. This is actually great because it means we can train and evaluate both without interference from chess players, possibly even building a stable society.
Well yeah... how would there be fewer??
But the point itself is silly. Few people are putting effort into Maths for the fun of it (and of those that are many derive fun from being the only one who can produce a solution). Chess differs in that it never had any point but the game its self.
this sort of lack of clarity is what drove people to learn more about the game.
like think about what it must've been like to go to a chess tournaments pre-engines: finals matches had everyone at your chess club watching the game, calculating lines with each other, and you HAD to calculate to understand the direction of game. it sounds so much more engaging and fun!
now, the top games have the stockfish bar next to it, and you instantly know who's got better odds without having to really follow along. and you're not encouraged to calculate beyond a few moves ahead because you just offload the real thinking to stockfish. honestly, it's a vibe a killer when you go back and see images and read about the culture beforehand.
I agree engines didn't really "kill" chess - and I think most of that is due to chess not being tied to economic value. but it did kill what I believe was a superior culture compared to today's chess era.
all this to say, I think what AI did to chess culture in the 90s is happening to stem right now with LLMs
I need to be very precise here, but it is not right to say that we know what the best move is in any chess board position. What we know is what move a chess engine would make that would win the game against a human, or sometimes another chess engine. We know a winning move; not the best move.
Winning in chess is not the same as knowing what the best move is. Having chess engines that can beat any human in chess is not the same as knowing how every possible game will develop. The latter is known as "solving chess" and we don't yet have that. We have fully solved games like tic-tac-toe, backgammon and checkers, so that there exist databases of every possible path through those games but this is not done for chess, and there is no hint that it is even possible to do given that chess is such a combinatorially more complex game than those.
More to the point, the upshot of the fact that we now have super-human chess engines that can beat any human in chess (and Go and shoggi) is that there is no longer the will to research the bigger question about solving chess. The solution of the minor problem, how to beat humans in chess, has sidelined and displaced research in the major problem, how to solve chess, and we will now never find the solution to the latter.
And if that reminds you of something, well, yes, exactly.
Genuine question: who were those people saying that? I don't know that criticism.
The criticism I know is from AI researchers and it is that beating humans at chess using a computer running an algorithm unlike anything that humans do when they play chess, tells us nothing about the way that humans play chess, which is what we are trying to understand when we try to get computers to play chess.
This criticism is exemplified by John McCarthy's article (the real godfather of AI; because he named it) "Ai as Sport" whence I quote:
Ideas about chess algorithms as well as advances in computer hardware were involved. However, it is a measure of our limited understanding of the principles of artificial intelligence (AI) that this level of play requires many millions of times as much computing as a human chess player does. Moreover, the fixation of most computer chess work on success in tournament play has come at scientific cost.
In 1965 the Russian mathematician Alexander Kronrod said, "Chess is the Drosophila of artificial intelligence." However, computer chess has developed much as genetics might have if the geneticists had concentrated their efforts starting in 1910 on breeding racing Drosophila. We would have some science, but mainly we would have very fast fruit flies.
https://www-formal.stanford.edu/jmc/newborn/newborn.html
A more recent criticism that can be made is that getting computers to beat humans at chess (and later Go, and Shoggi and Atari and Stratego) has turned out to only be possible with techniques that are too narrow to have real-world application. Btw, this includes Reinforcement Learning which is still much more capable in virtual environments than in any real-world environment.
AI will be better at digesting it into something more manageable.
"I like my job and want to continue doing it. It gives me meaning."
The brain then goes and fills in arguments that support these feelings.
So what went wrong? Mathematics education. Math below grad school is all about solving stated problems. Credit is given for solving puzzles successfully. Homework is problem sets. Everybody below a very advanced level is taught math that way. Even at the higher levels, puzzles remain important. Awards in mathematics are often tied to solving puzzle-like problems. That's still the criterion for becoming Senior Wrangler at Cambridge, "the greatest intellectual achievement attainable in Britain". This despite Polya's attempt at reform a century ago. Puzzle solving gets good grades and class rank. So it's the status indicator mathematics presents to the outside world.
Then reasonably good AI comes along. AI has become rather good at solving puzzles. So people aim powerful AIs at known hard puzzles, with some success. That blows up the status indicator system. Mathematics itself is fine. It's the status symbols that have a problem.
Maybe the Fields Medalists need to hire a crisis management team to reframe what success means in mathematics. That's what they're trying to do with that letter, but they're mathematicians, not PR people, and they don't know how.
They should explicitly advocate for the Fields Medal to weight teaching achievement more heavily going forward.
In general, in our existing society, status matters, and it is tied to achievements. Therefore a thing that devalues the achievement is also causing people to lose status, and everything that comes with it (income etc). Of course they are mad.
But maybe the answer to that is to acknowledge that our civilization is basically a dick measuring contest at scale, and try something else for a change? Starting with the part where people don't have to be "productive" to have intrinsic value as a person?
This is the crux of it and goes far beyond Mathematics or Computer Science. To get a bunch of humans to do anything, you have to motivate them. Kleos and Timē; renown and stuff. These AI companies threaten to rip this away from everyone but themselves, and this recent millennium prize is the perfect example.
Solving this problem as a human would have led to tremendous Kleos; my name would be written in the annals of mathematics, lecture tours of praise were mine to be had for the rest of my days. This one victory would have earned my recognition throughout history. The greatest a mortal may hope for. Ripped away.
It would also have given me great Timē. The prize money, the professorships, the book deals. Gone.
If all hope of “renown and stuff” in the intellectual realm is now taken by the AI companies, they will remove all human motivation to pursue these endeavours.
Perhaps the glory will come from slaying these fell beasts.
But there's still play. There's still curiosity. And there's still the drive to understand something for yourself.
Yes, but think about what that implies if those are the only motivations left. Gone are the professions. Gone are the ambitious.
There is plenty of space for people to work on intellectual pleasure pursuits (as there is with art and music), but the death of all intellectual based industries is still something to avoid. Or to mourn.
It’s an interesting view-point. What are the “right reasons” to pursue an intellectual discipline? To selflessly further the total sum of knowledge?
> No reason to mourn them leaving
What field? Who’s leaving?
The conclusion has to be that human involvement in these pursuits is facing existential crisis. “Teach me, oh Oracle, the mysteries of the Universe”. “For a fee”.
Both the glory and joy of discovery now belongs to the machines (the owners of those machines). Perhaps that’s fine and well, but I read Anathem and feel the strong yearning to be the intellectual monk. And yes, to fight and debate and think; and obtain my place in history. The history of man.
Yes. I think you are saying this sarcastically but I'm not sure... This is why you should get involved in research at the frontier.
"What field? Who’s leaving?"
Mathematics, and I'd extend it to other academic disciplines. Do you care more about the results and the knowledge or about who gets credit?
The drive for "glory" has many negative effects in academia that do not get enough recognition. Just look at the replication crisis in psychology for an example. The fact p-hacking was rampant among psychology "researchers" was well known even in the 90s. I remember my professors in grad school for physics joking about it way before it became a mainstream-ish story.
So why did this happen? Because there was no one with institutional power in psychology who both understood the problem and cared. No one who was interested in the furthering the "total sum of knowledge" rather they were interested in your "glory."
I do a lot of programming that uses mathematics. I designed the error recovery file format “par2”. I worked as a quantitative trader.
What do I want from math? To be able to search math results, understand them, and apply them. But academic mathematicians are so inwardly focused - as these authors so obviously are - that it does all of these badly! Search is still with words, not symbols and formulae. As for comprehension, many math papers are unreadable to those not in the cliche of mathematicians in that specialty. And even applied math papers don’t ship code or specify the operations explicitly.
I wanted to fix this and in 2013 I worked on formal proof. It makes symbol search and application easy. I found out that the tools had been usable for decades at that point. But mathematicians didn’t use them because they had that inward focus and didn’t give a damn about math consumers.
AI is enabling formal proof. It will carry mathematics into the computer age, which the field should have joined with every other industry in the 1970’s! I welcome it. Math will get used more and that is what the work was meant to achieve in the first place.
LLMs doing math has made math far more accessible to me than it was pre-LLM.
Not just using existing math, but developing new math. (For example, I developed an alternative to NURBS surfaces for geometric kernels using LLMs.)
It's really annoying that anyone, anywhere is talking about halting math progress with AI.
what the work was meant to achieve in the first place is what the work was achieving in the first place. it was developing mathematics. if someone wants to use it, they are free to do so, or to write a blog article how it can be used by non experts. mathematicians do that themselves sometimes, but it’s not their job. myb there should be more incentives for them to do more of such work
i keep seeing this myth that mathematics has some purpose to do this or that, but somehow its purpose is least known to mathematicians, everyone else seems to know better
Meant by who? Were the people footing the bill aware of this? People can like and do a lot of things. When money enters the picture it gets more complicated because usually the people who pay do that because they derive some gains from it. Historically scientists and mathematicians were employed by various patrons, sometimes doing astronomical calculations for horoscopes, Easter calendars and whatnot. Pure for-fun math was usually done by already wealthy individuals and not salaried employees like today. So it's fine that mathematicians feel this or that is their purpose, but they are coming to us asking for tax money, and so there is a bit of burden on them to be convincing. Before AI they used to argue how it helps physics, engineering technology, computer tech, optimizing logistics and telecom, cancer cures etc. Now the tune changed and it's retroactively now obviously just pure recreation and spiritual elegance. I also like math don't get me wrong. I think it's a great intellectual pursuit. But it's arrogant to expect that the money can flow by default.
In case of public universities, the mathematicians would ultimately have to convince the public, but of course in practice that means politicians and national grant agencies, who indeed report to politicians and the politicians chase popularity to the extent needed for reelection, so public sentiment has some relevance.
Generally when you come to someone holding out your hand asking for money, being humble is a good first step, instead of getting angry at people for not sufficiently appreciating the big myterious value of what they are doing. Explain the spiritual value of the pursuit, they may or may not be convinced. This may be called a deficiency of democracy if your aesthetic values are different, but typically people don't like to fund abstract disconnected things, like expensive space travel unless it's embedded in some tribal jockeying like during the Cold War. In normal times people just want research that has life enhancing practical outputs. Nevertheless currently the West is structured as electoral democracy, so convincing the plebs is indeed necessary, no matter how lowly that feels from the ivory tower.
what about people who study birds, or say, olms? people want that kind of research? the people who are doing it surely do. astronomy and eg star watching in deep space? i could go on for days
people like doing mathematics, similarly to studying endemic organisms or other stuff of no direct utilitarian value
how many parents are happy to pay the fraction of taxes that go into something that their children might want to choose as a calling and fall in love with?
it’s okay to discuss whether something should be publicly funded, using “people want x kind of research” as a supposedly obvious or common sense argument is fallacious in the sense that it implies an idealised steelman model of the general public. that model is divorced from reality
I got an economics degree and found that the City of Austin to pay off its $3 billion dollar pension debt … and encountered the same problem you mentioned, again. Except this time my plan is to run for City Council and be their next-level superior.
This is the effect of AI on most intellectual disciplines, and it’s a real worry.
This is a litmus test for the effective altruists in AI labs: fashion or conviction? The social pressure is to slow down and let the guild keep its norms a few more years. But what's clearly best for humanity is to push forward violently and leave them in the dust. A Fields medalist's feelings are not more important than the wellbeing of the planet.
They are depressed because what's being destroyed is mathematics, actually.
> Five years of guided computer-aided math could improve billions of lives or solve global warming outright through cheap clean energy.
This is complete BS, math doesn't solve climate change or save life, that's a category error. The purpose of math is to help us build understanding of the mathematical objects and that's it.
Sometimes, after much distillation to the academic world, this understanding is later being leverage by other scientific field, but that's a very rare event when it happens. And it's not how other science progress.
> This is a litmus test for the effective altruists
EA has always been a lie to justify short term individual greed by hypothetical shared future benefits.
his core argument that is LLMs (and specifically LLMs owned and gated by corporations was my read), while able to solve problems, do not contribute to this practice of knowledge building. solving problems is just one piece, and the mathematics community ingests problems and new methods, iterates and thinks on them, and then produces new ideas, methods, etc. this is what he is defining as progress, and solving things like millenium problems are markers of this progress.
But now, LLMs can generate hundreds of books per hour. They make up 80-90% of new arrivals in many nonfiction categories on Amazon. They short-circuit the system, allowing their "authors" to extract money from the system with zero effort by crowding out human work. And it's not even the question of whether these books are good or bad (although overwhelmingly, they're terrible). It's whether it's actually accomplishing anything worthwhile, or just destroying incentives for humans to write or go into any other sort of intellectual work.
In fact, I see many professions push back. Artists, writers, now mathematicians. And I'm amazed that our profession doesn't and that we have so many people who are hooked on vibecoding. I'm still waiting for that 10x payoff. All this velocity and somehow, the landscape of the software I want to use still looks the same as it did in 2021.
Is the fear post-apocalyptic in nature ? We need some human priesthood to carry on tradition ? why ?
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Just a thought.
Even in a hypothetical world where AGI can do everything that humans do better, this would still be a choice to make, not an inevitable consequence for everyone.
And life doesn't require suffering to have meaning.
I was saying that human pursuit of understanding of things provides value beyond survival and resources.
Joint pursuit of a common goal is a way for people to better be-in-relation-with one-another.
What it does is make it a choice rather than a survival necessity. The latter is where the "suffering" comes from.
I'd argue that this extremely extreme scenario is the only one in which it kind of makes sense to not have understanding. But let's be honest: no one knows if we'll be there (and it seems unlikely since everything reaches a plateau eventually). So, what happens if we allow ourselves to forget everything and then we don't reach the ideal scenario?
The hopeful note is that I do think we are entering a golden age for the curious casual/semi-pro mathematician and for niche mathematics areas that won't get the attention of the top labs. Everyone is sprinting to solve the millennium problems, but this is a very exciting time to be in a sub-sub-field where you and 4 others are keeping things alive.
Now AI has surpassed me in fine-grained problem-solving ability. It can write more reliable code than I can, faster than I can, provided it is given the right guidance.
But one thing that I'm still much better at is identifying technical opportunities and choosing the right tradeoffs.
My feeling is that frontier models are incredibly smart in some ways, but incredibly dumb in other ways
When I chat with Claude using deep technical language about distributed systems issues, the arguments it presents are mind-blowingly good. I worked with many skilled engineers on complex projects but the kinds of arguments Claude makes are on another level. It feels like it can read my mind because it already identified all of the relevant aspects to the current topic and it's incredibly persuasive. I would say it's even better at deep, nuanced technical discussions than I am.
When I ask it to implement a feature using technical language, it does a really good job and the solution usually works out of the box. No bugs at all 95% of the time even after adding 1000 lines!
But the problem it still has is that when it implements solutions, it misses so many low-hanging fruits/opportunities. Especially in terms of performance, maintainability, scalability and UX. It's like it sees and recalls all the relevant parts perfectly but yet somehow misses opportunities which seem extremely obvious to me.
It feels like AI has 0 creativity. It only sees the opportunity once I mention it... And once it sees the opportunity, it demonstrates deep understanding of the technical implications. I think what's surprising is that it understands the suggested solution so well, with such nuance, that I can't understand how it didn't see the opportunity and why it never seems to see it until I mention it.
This is very unhuman-like. There is no way that a human being with that degree of understanding of a topic would be presented with such highly relevant context and not make the connection.
This situation suits me fine, since I am anyways more of an ideas person, than a crunching open problems person. But I understand the desperation of my colleagues who mad solving hard problems their identity.
That's the real benefit of AI today: you can test your ideas almost as fast as you can generate them, in parallel even. If you have a crazy idea that is unlikely to work but has massive advantages if it does, you just set an agent to explore and prototype it while you focus your main attention elsewhere. And so on.
It's still GIGO, though! An AI can make a bad idea work (and will do so if you don't carefully prompt it to not be so sycophantic!) but it won't work well. Knowing when the idea is bad and should be abandoned is also the part that currently requires human judgment.
That said, given the pace at which models have been improving so far, I can't help but think that this is a transient state of affairs.
The people who prided themselves on raw puzzle-solving ability got hit hard. Those who are idea-driven and architecture-oriented feel like they got handed a superpower.
It's quite a big shock at the industry level because the entire software engineering job interview process at essentially all large companies was heavily biased towards well-defined, raw problem-solving under time constraints which is precisely the skill which AI has replaced.
You're right it's very unhuman to miss obvious opportunities you mentioned. And what's why I think these models would continue sucking as long as they don't have human level general intelligence. They may get marginally better at these tasks but I don't think we can expect them to connect dots like humans before AGI.
It seems like a lot of the issue here is that these problems aren’t interesting in and of themselves, but they lead down interesting roads. It defeats the purpose if you solve them without getting any real understanding.
It’s akin to saying you’ve solved “pancake flipping” problems with a waffle maker, or “travelling salesman” problems with a zoom meeting.
It was never about helping individual mathematicians demonstrate that they are individually good at math. It happened to work out that way, but it wasn't the goal.
Well, one thing is stopping them. There will be no more adoration for their genius.
If you truly do it for understanding and not the attention, carry on. AI should change nothing about your motivations.
My understanding is they are? And literally everything in this world is based around incentives. If you say “well you can continue to work on understanding, but your kids are going to starve” that’s not nothing.
We absolutely want to, of course. But you extinguish an industry and the systems of training that supply it. It's hard to know if letting it go that way is right.
I have a the cure for cancer. Simply kill the host. Does it work? Yes. Have you learned anything from it? No.
AI should be the commons.
People like Grigori Perelman would baffle you, a mathematician who solved the Poincaré Conjecture, refused the monetary prize, field medal and continues to live a life of total recluse.
For most mathematicians their primary drive is chasing the unknown, not for anyone’s adoration, but to pursue their desire to see what lies in the beyond.
Keep in mind employees at AI companies are publicly stating that they believe they're risking a >10% chance of human extinction. They're knowingly risking the lives of every man, woman, and child to continue the work. The lives of their own sons and daughters. A person already rationalizing that isn't going to shed a tear for the careers of mathematicians. Just a bug on the windshield.
What got me interested in memes as a kid is precisely the fact that mathematics is true in a way that is wholy independent of our understanding of it.
Are we really going to take what they say in public seriously?
Ironic or what. Mr Tao may be remembered as Mathematics' Canute.
AI companies are alienating the communities they serve. Instead of a win-win dynamic, they are keen on a win-lose proposition. You dont win trust by one-upping your customer. This is unfortunate and suggests a lack of adults in the room. It also reeks of hubris and is all good when making profits is not a concern. But watch the narrative shift when there is an AI slowdown which is inevitable.
Of course, nobody's really comfortable with it, so this is also a forcing function for the industry to adapt and figure out new techniques to manage complexity and trust. I think the same will happen with Mathematics.
But it is also possible we will end up with three forms of Mathematics: the one we understand, the one we don't, and the one we don't understand but can prove to work. Kind of like magic -- with all the positive and negative connotations of the word.
It is pretty evident that these models will soon exceed our cognitive capabilities. Is it right to hold them back just because we can't keep up? Many of those discoveries will be so beyond us that we can't do anything with them, but that also means they can't hurt us. On the other hand, there could be many discoveries that we can parlay into practically useful applications, even if we don't understand them.
Just like LLMs.
Yes, this has gone so well
That's not true, many of these outages have been directly attributed to AI tooling.
> The rest of the world has been happily chugging along with coding agents for almost a year now and things seem to still be working just fine.
Many more services are now being attacked by AI agents that originate from all kinds of organizations including OpenAI, Anthropic, and many others. Unless you're purposefully being obtuse, I would not call that "working just fine".
- Except the mathematicians who we'll scoop and cause existential dread among their entire field.
- Except the software developers. They'll need to become plumbers or live on UBI.
- Except the people in countries that can't afford the cost of AI tokens to keep up with the rest of the world.
Just keep picking off groups of humans for the "benefits of all humanity"... while building larger and larger disparities been the have a lots and the just have enoughs.
We're going to build humans a utopia but along the way we'll leave a trail of destruction because that's not our problem.
We could imagine, as an extreme case, a technologically highly advanced society, containing many complex structures, some of them far more intricate and intelligent than anything that exists on the planet today – a society which nevertheless lacks any type of being that is conscious or whose welfare has moral significance. In a sense, this would be an uninhabited society. It would be a society of economic miracles and technological awesomeness, with nobody there to benefit. A Disneyland with no children.Would you argue it didnt benefit humanity, because taxi/bus drivers are nolonger required
If all that AI brought resulted in just taxi/bus drivers being phased out of their jobs in a thoughtful way, then that would be more manageable at the society level. But we're talking about almost all sectors of the economy.
If the magnitude of changes that OpenAI and Athropic believe will be delivered with increasingly powerful AI (and robotics) comes in a time frame that significantly worsens a large proportion of people's lives, this is a different situation. Can super powerful AI not be developed in a way that minimizes such disruption?
Remember when we all said it's a good thing when the coal mining jobs are going away and that they should all just learn to code? Maybe a little more of that energy right now.
It cannot. The problem is that everything gets automated, so there's nowhere for people to "transition" to.
More fundamentally, the problem is that capitalism is simply incompatible with this. Specifically: capitalism is a system of property rights which in the long run always results in concentration of capital in a few hands. What made it tolerable so far is that capital cannot produce wealth by itself - you need people to use it to that end. Which in turn means that you need to pay for labor, at least enough for it to sustain itself. So even if you pocket most of the wealth that is generated in the process, the workers still get the crumbs at least.
AI is not like automation in the past because its end goal is doing everything that humans can do. Taken to its logical conclusion, you get capital that doesn't need labor at all: those who own the robots live in a personal post-scarcity utopia, everyone else starves. A slightly better case is when everyone else gets a meager pension, just high enough that people have something to lose and don't try to burn everything down - this is why you hear so much about UBI from the likes of Sam Altman (this isn't to say that UBI is inherently a bad thing! but make no mistake, what they actually want would look a lot more like The Expanse and a lot less like The Culture). It "minimizes disruption" in a sense that there are no riots, and it might even be a material improvement for a lot of the globe, but it sure isn't an improvement for most people living in developed countries today.
So I would argue that, if anything, we need more disruption here, not less. All the way to the top.
1 + 1/2 + 1/4 + 1/8 ... = 2
The big objection to finiteism is that it's a lot more work. Infinity swallows many special cases. Proofs get longer without infinity, and most of the special cases are uninteresting. That's not a problem for AIs.
Someone may start up an AI and make it grind through Hilbert's program for putting mathematics on a fully consistent foundation, starting from a finiteism base. This is a huge, unrewarding job. Great for machine work.
It's not necessarily clear that this statement requires infinity, if you're willing to treat "... =" as a shorthand. You might prefer something like "1 + 1/2 + 1/4 + 1/8 ... -> 2" if it's more clear, where "->" means something like "gets as close as you like without ever getting further away than that", but really the "=" sign is already overloaded in all sorts of subtly different ways anyway, so there's not really any trouble using it here. Almost any rigorous definition you can write down of exactly what that statement means would not rely on the use of infinity.
If you allow infinite recursion, you soon get to Godel and undecidable problems. Finite deterministic systems are decidable, because you can in principle enumerate all the states. The halting problem is decidable for deterministic systems with finite memory. It may be exponentially hard for some programs, but that's quite different from being undecidable.
(This is too long a subject to discuss here, and I haven't worked on constructive mathematics in many years. It's more practical than it was decades ago. You need power tools, which we now have.)
A better example would be a limit that equals sqrt(2) which finitists would probably say cannot represent a real object because it is only defined as the end of an infinite process.
I totally see the problem Terence is describing. We are loosing a lot in understanding and focus if it continues like that. The solution found for Navier Stokes doesn’t have much „real value“ - but what almost always happened in the past when people worked on the difficult problems, these sparked new ideas / new theorems that broadened our knowledge. Think back at your grad studies, figuring out a proof as homework was hard, sometimes incredibly hard, but while doing it we gained a lot of understanding how things work. Now asking AI for the solution and „just“ getting it, risks our understanding, our creativity and our ability to connect the dots with other territories. I see it in students nowadays, there is much less understanding, much less creativity in finding solutions. I truly think this „short-path“ solution with the „death of struggle is one of the biggest risks with AI already for human development
If the complaint is about AI just giving answers and not good understanding around them, or frameworks that lead to more solutions from them, then that means there is a gap that can be filled by human mathematicians.
So all the AI has done has actually made mathematicians lives easier and provided them with an opportunity to fill that gap. They should be using it to do that, instead of complaining.
In fact, I would bet if this gap were not present, it'd be an even worse complaint: Now we have absolutely nothing to do in the field.
What's the point of being human if we dont do human things but entirely rely on AI?
I believe this is more or less these mathematicians' argument.
Research is fundamentally different. We are seeing the erosion of specific needs for thinking at depth. AI is the automobile for the mind. There will 100% be undesirable consequences and selective atrophy of cognitive abilities once prized. This is a loss. There's no getting the cat back in the bag at this point so long as the objective dimension of work, as in object opposed to subject, is held as the most important.
SWEs felt this same crisis late last year. Now it's the mathematicians. They won't be the last.
Philosophers I'm sure can debate this back and forth but it seems probable that some things we thought were ineffable are in fact quantifiable to a degree, and now we have the technology and the machines to bring that to the logical conclusion.
The answer may sound harsh, but we will not need human once AI reaches that threshold. Current IQ of AI is currently 130 as per Google.
[1] https://www.slatestarcodexabridged.com/Meditations-On-Moloch
(Edit: assuming you are human)
Which is certainly how our society is structured, but, does it have to be? Can we admit that we basically have a developmental trauma as a species and learn to live the sake of living, instead of for the sake of being "useful"?
There are too many uncertainties around who will be in control and with what kinds of aims.
It would end in some kind of prison or zoo.
It's fun.
> Of talking to one another?
It's fun.
> What's the point of being human if we dont do human things but entirely rely on AI?
It's fun.
Well, it's not fun today because most of us have to spend most of our waking hours working some bullshit job for a living. But imagine that there's no bullshit job?
This is about how good taste in both research direction and in design are essential to steering AI, but we have no plan at all for instilling that taste in students or practitioners in a post-AI world.
> The issues the mathematical community faces now are similar to issues that other scientific and creative professions are facing, and indicate issues that all of humanity might face: how to make sure that, as AI changes the way work is done, we do not lose sight of what that work was meant to achieve in the first place.
Besides eroding taste and taste-building, this is about just how useful friction is as signal.
Everyone coding with AI knows it routes around difficulties like a river around a stone, which is not necessarily a good thing. It will do it tirelessly 1000 times instead of learning anything from it. AND if the AI does not fail in this, the human driver will get no signal, and never know it happened. This seems to be getting worse, not better.. my theory is that more models are cross-trained on cybersecurity stuff where the goal is success and the method doesn't matter. Fine for pen-testing, ultimately pretty bad for coherent code or math or physics.
Discrete tasks where we don't want to be bothered is a real use-case, but optimizing for it everywhere is terrible for the future of durable abstractions that we can build on and ratchet up our understanding with. Bad for the models too eventually! They can maintain a codebase with millions of special cases or juggle tons of free variables in equations, but that just encourages bad abstractions.. they have a ceiling for this too, even if it's higher than humans.
In almost any scenario even tangentially involving mathematics, twenty-five Fields medallists uniting to denounce something would be a veritable Tsar Bomba.
It should give you pause that here they feel like the ailing infant.
>We candlemakers are suffering from the unfair competition of a foreign rival. This foreign manufacturer of light has such an advantage over us that he floods our domestic markets with his product. And he offers it at a fantastically low price. The moment this foreigner appears in our country, all our customers desert us and turn to him. As a result, an entire domestic industry is rendered completely stagnant. And even more, since the lighting industry has countless ramifications with other native industries, they, too, are injured. This foreign manufacturer who competes against us without mercy is none other than the sun itself!
>Here is our petition: Please pass a law ordering the closing of all windows, skylights, shutters, curtains, and blinds... https://fee.org/articles/the-candlemakers-petition/
The petition is about a much broader problem: if we’ll demotivate people from understanding math, vibe math in a rush without making it understandable - math becomes useless
Basically with pic generation at least we all understand what is on a pic - some cat or something
With math it is trivial to generate a lot of stuff that is half-assed and just gives a plain yes/no but doesn’t get absorbed into human technology and better math
Basically it is not inly about the answer 42, it is about the questions, _meaningful_ _full_ answers and empowering people
By giving a short answer AI can burn the find and make it boring, hide the fertile ground around that answer that any human would’ve highlighted
If you have Einstein - it is not just relativity he did - you can continue asking him questions and he continues the research direction, teaches students, etc
With AI it is often 100% gate keeped (most of the time experimental AIs do the stuff, you have no access even for 200 bucks/month or more) slave that was killed after finishing - hard to “compete” with that and it is not very inspiring approach to researchers ;-)
It answered about the Navier Stokes too but not at a very deep level https://share.google/aimode/JD69g15GIQiYo9L65
Ah yes, Lean 4. Famous for how unreliable it is. Absolute slop!
I wonder if there’s a Fields Medalist group chat.
The process is as much a human experience as is the result. If we have dime-a-dozen results like this minus the process, where does our intellectual structure stand? We will just be a hierarchy of priests at the altar of the machine?
PS: Before someone comes up with an counter example of maths before and after calculators, that was a tool that accelerated the steps. This is replacing the brain.
Sure, but in your scenario, the process is the experience for a small group of people, while the result benefits everyone (perhaps not directly, but ultimately via a chain of applications). Which one is more important?
It's not at all a new dilemma either. Jacquard looms destroyed the "human experience" of manual weaving for craftsmen who spent decades to perfect their skill. They also made clothing orders of magnitude cheaper for everyone else.
They also contributed to consumerism and use-and-throw culture and mounds of waste around the world. People were not mortgaging their homes to buy clothes. There was a spectrum.
Also, your example is of physical skill. Mechanized farming has lifted many out of hunger. All automation isn't bad.
However this is the last frontier of higher pursuit for mankind - using our minds. This is a totally different level.
OpenAI/Anthropic are shaking the box.
Don't worry about what they write, they just want to feel emotions from any news article.
Mathematics is fine and will be fine in the future.
Everything is sensationalized, every super niche happenstance is sold as earth-shattering drama, the outrage arms race is so tiresome.
Man, the people who want to just get away with open corruption sure love you.
"Everybody's enraged, why don't you like this unethical thing being done to you by a company?"
What's going on here?
If the worry is that AI companies are turning open problems into benchmarks and potentially “using up” fertile mathematical problems before humans can develop the ideas around them, what exactly should the companies do differently? Also why does discovering the answers preclude humans developing ideas from them? I don't get why solving a math problem stops anyone from doing that?
Should they (AI companies) avoid training or evaluating models on open problems? Solve them but not publish the results? Delay publication? Only release proofs after mathematicians have had time to study them? Require some attribution or review process?
The statement makes a strong case that “maximize the number of solved problems” may be the wrong objective, but it seems much less clear about what behavior they actually want from OpenAI, Anthropic, DeepMind, etc.
I’d be interested in the most concrete version of the proposal. Without that, it starts to read a little like: "Please stop getting so good at our thing!"
Just stop doing that. Don't treat unsolved math problems as some cheap benchmark to beat.
Leave the math for mathematicians, and let them use AI in a way that helps the field, not in a way that harms it.
To Tao’s credit he obviously identified the problem very clearly and admits understandably "we did not have the time to have a more consultative process, as with Leiden; but we decided that the urgency of the situation was such that we needed to release a statement sooner rather than later".
Why? If someone makes an innovation that undercuts the underpinnings of some existing institution, why are they are responsible for cleaning up its failure?
Grothendieck was anti-slop but most papers are slop.
I don't think AI is going to rewrite bourbaki anytime soon
Scholze's math is definitely not slop.
But you're taking two of the best mathematicians of the last century against my claim about averages
Out sourcing construction jobs was great for the economy while leaving entire cities in rubbles.
But as soon as it hits the privileged class there is a call to "provide a specific replacement mechanism".
Older people are desperately trying to keep a grasp on their current power and lifestyles at the expense of younger people and technology.
We need to ban Waymos because taxi drivers need to be protected.
We need to block housing because it would lower my property values, and eliminate property taxes while we're at it! I don't use the local schools so why should I be taxed to pay for it.
We need to spend recklessly to pay my pension and have the next generation foot the bill.
Its just a repulsive ideology.
We apparently have a moral obligation to protect existing power structures?
Tao should maybe consider there are people who are indifferent to, or actively want to tear down, his institutions; why should they cooperate in preserving them? Whatever happens has to be resilient in the face of defection; any scheme where everyone is expected to agree to not use AI in a way he doesn't like will not qualify.
I think he's in the "bargaining" stage of dealing with loss right now.
not sure it's comparable, but the issue is that for a lot of those mathematical results, they don't really have utility by themselves. The utility is the new branches/understanding that's being developped.
So, why can't they just be ignored?
Alternatively, some claim that mathematics is about understanding these implications.
Under the first definition, AI is already, and forevermore will be faster and better at proving theorems. Just like it is better at checkers, chess, and now go.
The author asserts that AI proofs are incomprehensible to humans, and so under the second definition AI is merely a tool to overcome one hurdle on the way to understanding.
So which is it? The author seems to claim the second definition, but bemoan the end of mathematics under the first.
That's like saying that programming is about producing valid programs in various programming languages.
But then who decides why a statement is mor important than another? In the eyes of a formal systems all statements are born equal.
Some are more useful, and that is something you can quantitatively ask.
That sounds like an "us problem", not an AI or OpenAI/Anthropic problem.
I don't mean to be a dick, but I've talked about it previously. These folks are grieving. I get it, I've lived through this sort of life changing thing before, it sucks... but yeah.
I am a bit disappointed by him.
Dismissing it as "innovations have happened before" is disingenuous. Yes, innovations have happened, but none of those threatened to automate all human work in existence.
Our society has functioned very differently in the past, precisely because the technology level made some things viable (or profitable) that aren't so anymore.
One would hope that, rather than abandoning technological progress, we'll rather adjust our society to it.
I mean, seriously, read what you just wrote. You're literally saying that solving scarcity is a problem because of the way our society currently functions - because people "need jobs to eat, pay for housing, etc". And you're correct! But what does it tell you about our society that robots capable of producing everything that humans might need is a problem for it?
My read on this document is that people's work isn't being fairly cited more than what does it mean to be a mathematician in this age.
> Often these solutions are announced in a rush, leaving no time for a proper writeup, the isolation of new methods and ideas, and citing relevant previous work of others. As in all creative professions, this raises severe attribution and plagiarism questions.
Good thing they never did that then
We are all going to have to come to terms with entities more capable than we are, and in many cases, letting the real work be done by the AIs will be the right thing to do. For all the huffing and puffing about the "human touch" in medicine, it will eventually become downright irresponsible to consult only with a human doctor. I am not sure if this is the case in mathematics or not, but if it isn't, that suggests math will be relegated to more of a hobby than a cutting edge scientific discipline.
If you spend $20M working out a Millennium Prize problem, in what universe would you offer that an unrelated effort should take credit? This is a branding game rather over whether software engineers are going to use codex or claude. In that light $20M (or whatever it was) might be worth it to squash even the rumor that claude code is more capable. Engineers look up to mathematics, while at the same time business and probably most engineers think the problem was to solve the problem. GPTs solved one the hardest known problems so they can solve my company's problem.
Some go further looking at these people, very on-the-nosely likened by one commenter here to ants, talking about education and responsibility and "core values" etc and just don't care. There was a major problem at the beginning of the week that is not a problem now and that is uncomplicated progress.
It's not wrong for OpenAI/Anthropic to do math for product development or even just branding but seemingly at no cost now they could work in an arena real mathematicians aren't interested in, versus just mowing the field. Everyone involved on their side should admit the purpose of these demonstrations is not to engage mathematical ideas it's about Claude/Codex. there's no shame in that. Which is better at solving random hard Diophantine systems? That would seem to tell me as much as I need to know insofar as a model's value is represented by raw mathematical power - then, take my money just as well!
Assuming the worst accusations are not true I think there are ways forward going to be acceptable for all. The labs themselves do not represent Terrance Tao as some kind of gate-keeping dinosaur in this. They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes are paid 7-8 figure salaries ultimately for the product, they solve a Millennium Prize problem then pretty quickly seem to move past it.
Well. Obviously, yes. But in doing so they still DID scoop out a mathematician in a very unethical way.
In doing so they showed that they basically have a huge gun they can point at X work you care about and develop, and can cut you across the finish line. And take credit for it. Obviously this already _existed_, but is just much more significant because even a rumor can be converted into a complete takeover of a discovery.
> They're not interested, not the kind of entity that can care about theoretical mathematics. These dudes take home 7-8 figure salaries, they solve a Millennium Prize problem then pretty quickly move past it.
I get your point, they don't really care about solving all the maths problems. But they're still going to solve them for clout and profit motives. Up until it stops wow-ing people... at which point they will have likely decimated the frontier of the field.
And this is kind of the root of the entire concern. They will move into the forest and completely steamroll all the problems, then declare victory and move on, leaving only pavement and asphalt behind.
In the age of AI, there's no reason one has to follow the kind of classes like Algebra, Topology or PDE. Teach just enough so that good students can understand the basic, and go straight into seminar and research math. I don't think a top student in sophomore year cannot understand or work on some combinatorics research problem and get some results, with proper mentoring and guidance.
Beyond the issue of growing understanding and keeping a bountiful stock of questions to pursue, this scheme seems to be threatened as well.
Huh. Weird. This hasn't been my take of mathematicians at all. The dozens I know are quite humble and dedicated to math and the beauty one finds in it.
It is true that the manufacturing of "true/false" statements is not the same as gaining understanding of a problem. However, for many mathematicians, true/false statements are already manufactured by others. Think of a student who is given a conjecture to investigate, with their advisor describing it as "it must be true". Most exercises in a textbook are stated such that the outcome is known before you begin. That's not really a problem -- investigating the conjecture/exercise yields its own dividends, whether or not the outcome is known. It is also the case that defining new directions involve understanding and synthesizing related problems, asking the right questions, and deciding upon the right directions, and it's not clear that AI can do that at all.
The real risk, I think, its the public's (and funding agencies') perception of the importance of "human" mathematics, but that's already a struggle. For example, it's tough to explain to the lay person why it's still important to research group theory -- the main example people cite is RSA encryption, which was invented almost 50 years ago.
Frontier labs need these headlines not for human progress but as beauty pageant for investors and government agencies. If they don’t do maths they’ll just go after other fields.
So Terrance Tao here might be able to hold them off math but he won’t stop them from speedrunning STEM with similar consequences.
We may be locking people out of these fields instead delegating everything to machines, and I don’t think the machines are good enough to assume that responsibility.
I don't care how good Astra or any subsequent models they may release might be... I am never going back to those token reset shenanigans.
There are a lot of us who just use the AI on projects until the session limit hits, and wait for the usage to reset.
Do you mean that you can't continue on your own until the reset?
I went from using it non-stop all day every day for months, to running into my weekly limit within 24 hours almost overnight.
They lied about token efficiencies and everything... said they had no idea what the problem was, etc... and then bam, once China starts releasing more powerful models, they start "resetting" our token limits constantly ... sometimes ... maybe ... if we're lucky ...
I am over it.
I don't care WHAT I pay to be perfectly honest. I would have gladly paid $2,000 per month for the service I was receiving.
I just don't like being jerked around like that.
Toodles, OpenAI.
Imagine time traveling back in time and offering Leibniz a packet of proofs from the intervening years, but with the caveat that there would be no explanations. Would he say no?
And that’s the issue the article is trying to explain.
We are moving forward and if that means no human wins a fields medal because they didnt spend three decades working on a problem that could be solved in three days, the world will be better for it.
Once AI mines the easy unsolved problems, it will take large token budgets to make meaningful advances. That will need to be funded and planned at a societal level, like the great facilities of the other big sciences.
Mathematics will become more like an observational science, with some mathematicians specialized in interpreting the results that come out of the big AIs.
They spent millions to solve this problem but it’s not like someone is going to pay that for every problem
Won’t progress still be limited financially?
There's a separate question of who controls how much energy, and can therefore decide where it's spent. This isn't an inherent limitation to progress, though - it's an imaginary limitation that exists in our minds, a collective delusion, if you will. There's no law of physics that says Sam Altman gets to decide; we as a society choose to respect the property rights that translate to his economic (and attendant political) power to do so. We could choose to stop respecting them, and if we do so collectively, there's nothing they can do about it.
These problems are just for marketing for the time being
Maybe the worries of the field of math being “over” are a bit overblown since a lot of pure math these things may discover has no return to the company
I do see a patient teacher on generative tools. What has keep me trying to learn math has not been mathematicians, but the support of people outside of the field.
Total horse shit, but that's the essence of it.
An "unsolved problem" as a beacon to strive towards, gaining understanding along the way. With the beacon dead, noone will strive.
I'm unclear what the ask is, though. What, even in theory, is a practical and realistic fix?
On one level, they admit that finding solutions and generating understanding are non-overlapping magisteria. They use this admission to make the point that AI models are not generating understanding by finding solutions.
But on another level, they assert that AI finding solutions somehow breaks the process of generating understanding.
In other words, they simultaneously do and don't understand the non-relationship between finding solutions and generating understanding, and they choose which mode to operate in depending on which helps them score a point against AI.
The only reason we have historically perceived a link between finding solutions and generating understanding is that the problem space was larger than human cognition allowed us to search efficiently. So we pretended, in a "security through obscurity" way, that finding a solution implied understanding. It's inconvenient that we can no longer do that, and institutions will need to adjust in response, but it doesn't break the foundation of mathematics.
But I agree with the sentiment that the marketing behind these "discoveries" is disingenious. They pretend they solved the problem, but it still takes a bunch of humans to reduce the solution to a simplified and sensible explanation.
The fundamental issue with AI solving any perceived difficult problem is that we have lost the journey. The sight atop Mount Everest looks much different when you have climbed compared to being dropped from above.
So that raises the question: is mathematics simply a pursuit of passion? Are problems solved "because they're there"? If so, then mathematics can join the ranks of things like mountain climbing, cycling, and weight lifting. But if we are trying to accomplish something important (design better airplanes, find theoretical guarantees about cryptography, factor matrices faster), mathematics needs to become more like a military or search and rescue operation, using the best technology available to secure the outcome we need. Given that the NSF pours billions into scientific research every year, it sure seems like mathematicians want to think of themselves as being in the latter category.
If AI gave us the plane to reach Everest without us having gone through the journey of aviation and flight, what would we have lost without that process?
But the most important problems to be solved are not technological challenges but social ones, involving humans and our relationship to one another. An area AI will forever ill-suited to handle.
This is true of almost any field - if we have black-box technology that only the AI can understand / fix / augment / extend, then we will lose the ability for humans to learn and understand that tech.
Its similar to outsourcing manufacturing, you lose the ability to manufacture. If you don't exercise a muscle, it atrophies. If some people dont learn to write code [ or do math or write music or books ] the hard slow way, we will lose that deep skill.
If young people don't grapple with hard problems in science, math, economics, creative writing, drawing, music making .. they will not be trained, their cognitive skills will not develop with use. We will have no doctors or engineers or musicians or lawyers in a decade or two.
We don't want to become a race of fatties who never exercise, or dummies who just scroll social media feeds without exercising their minds.
This is real.
I wish they had stayed this as the very first paragraph.
Maybe I'm too stupid to grok the original declaration. Does anyone have a better summation?
As a non-native English speaker, I initially understood this to mean that all living Fields Medallists had signed. I later realized that it meant only that all the signatories were Fields Medallists.
(Apparently, there are 47 living Fields Medallists today.)
> ... 25 initial signatories — all of them Fields Medallists —
OR
> ... 25 initial signatories — all of the living Fields Medallists —
The first one is what they meant.
Every one of the mathematicians on this letter was likely aware of the progress in AI, and knew this was coming if they were honest with themselves. Some have even helped pushed the agenda forward. Inevitably, there would be a day when AI would find solutions to problems that defined the field. That day has come perhaps sooner than anticipated.
This letter smacks of fear. Their way of learning, work, achievement, and community is threatened by the ever steady pace of technology.
They say:
“But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of "true/false" statements could destroy fertile ground instead of breathing life into new ideas.”
Indeed, but the mathematics community needs to look inward not outward.
Achievement has long been defined by solving problems that other mathematicians and scientists have found crucial for understanding shapes, numbers, and natural phenomena.
The race to solve these hasn’t always been collaborative, cordial, or healthy. Too often, individuals have worked in secret or in small groups in order to claim priority and the accolades that come along. Further, I’ve even seen worse behavior not worth getting into here.
Now AI can solve these problems, oftentimes without building the abstractions that we need for understanding. And it is increasingly easier to collaborate in large groups without needing to trust everyone’s ability because we can check the truth of their work and incorporate it into the larger effort.
Certainly, the mathematical community and AI are misaligned. But, it is unclear that AI needs to change. Rather, I argue, it is the mathematical community that will need to change. Instead of focusing on the teleological pursuit of the truth of blessed problems, we need to reward the concise explanations and beautiful theories that give clear understanding. In turn, we can use AI to help us collaborate better and better our understanding.
This is Terry Tao talking about AI's impact on Math, but this could just as well be a software engineer talking about AI's impact on software development.
Do mathematicians deserve more job security than software engineers?
The threat to mathematics isn't that suddenly the profitability of their profession (lol) is going to go away, it's that people are thinking of AI as a replacement for the human social and intellectual practices that constitute the discipline.
You could say the same about software development.
Software development is a group effort, so it includes social practices, and certainly includes intellectual practices as well.
For the sake of argument, how is this different from the Luddites? The Luddites feared that machines would displace not only human labor, but also the social knowledge, skilled judgment, and craft traditions embedded in their work.
Perhaps you are misaligned.
Who decided the goal of math must be human insight?
First off, some mathematical truths might simply be far beyond our biological comprehension.
Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
Isn't this why we have math in the first place? To solve our real problems? Over time it morphed into this pursuit of pure theoretical insight, probably out of necessity at the time, but is it still necessary?
This is just the problem with the Navier-Stokes solution. We now (supposedly) know the truthiness of the statement, but we have no more understanding of why or how it applies to our quality of life. This is exactly the issue Tao is pointing at.
By analogy with code, do we still need code to be maintainable/readable if machines write it all?
(Obviously for now the answer is yes, but I’m not sure this will be the case in 5-10 years)
Now, I can ask ChatGPT about this and get back a proof that shows "a passive airframe cannot sustain zero-drag motion through still, viscous air"
So, I think if anything now, Maths has changed for the better. More ideas can be proven false or true from a get go instead of wasting so much to see if its even feasible to find out it isn't.
Progress if anything is about to leap frog anything we have ever known.
If you are suggesting there is some new model of physics or groundbreaking technology that would allow such a dragless plane, then don’t let me discourage you! But AI won’t help at all, since it will only regurgitate conventional wisdom…
A program is not the only output of programming. The other, arguably far more important output, is the programmer.
When you write the program — with your own hands — the program is proof that you have a solid mental model of the program.
When you let the computer write the program for you, the program is proof of … nothing.
https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...
But nothing’s stopping anyone to still work out an alternative proof, or a more elegant proof, or just trying to prove for the sake of understanding, just like doing homework without looking at the solution. It’s just that you can’t get paid doing that anymore.
Let's assume in the next decade GPT-7 PRO Ultra is cheaply ubiquitous, inspectable, reproducible, transferable, reasonably un-constrained by any institutional interests.
What say the 1% ?
Lifted up my comment for addition visibility
The second is the utility. We get to do navigation because the maths about angles and spheres checks out. The social utility downstream of AI proofs may be huge.
I don't like this disjunction.
I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.
But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.
Alignment alone is not enough. Aligned with _what_?
Without the ability to do things the "hard" way it is difficult to figure out if doing things the "easy" way will help us advance the frontier of math and science.
I may be wrong but historically we had this version of science discovery for a long while (empirical observation and brute force application) rather than first principles leading to applications (tools, the wheel, mills etc). Then somewhere along the way it flipped after Newton and the enlightenment period and started understanding first principles before they become engineering applications.
Perhaps it is not required, and we can just keep doing things the "easy" way like we used to, or we might find ourselves out of the ability to brute force things and then we go back to needing to do this the hard way, at which point this period of AI brute forcing would be seen as a detriment.
Physics and Biology will see major breakthroughs that WILL fundamentally alter our world. That is one key thing missing from alot of discussion here is the narrow focus on math (or parallels with software engineering). Doing well in math is key to doing well in physics and other sciences.
The big difference is my negative is concrete, possible and enabled in 2026 by GPT Astra while you’re talking about some future which we have no idea, we will ever reach.
Protecting math jobs limits cancer progress even being a choice of what to pursue. You could choose to allow 24/7 surveillance, but why would you? At least with math progress society can choose what to allow from it, but slowing it down universally blocks all choices.
It sucks that our democracy is really mostly just for show and thus such measures are required, but yes, we absolutely can decide such things if we really want to.
I get that mathematicians and other academics have their social structure and mechanism to create social hierarchies but those aren't very important to interest of general population that funds them.
If the process of making sense of the results, finding structure that humans can understand and writing explanations that allows for wider understanding is valuable it will still be valuable and there will be need for people able to provide it. If it makes higher mathematics accessible to more people then it's even better. I am hope they will not manage to sacrifice that for the sake of their little society in society.
Train an LLM with no advanced math texts: only basic math up to 6th grade, conversational text and literary works.
Interact with it (you cannot refer to anything past 6th grade math since you don't know it yourself) and get it to propose a solution to a real world problem. e.g., come up with RSA to practically secure communication.
Seems the same to me. And it'll be the same in all industries soon enough. And then it won't just be the junior people.
All the same problem: what do people do now?
This is still happening.
“I say your civilization, because as soon as we started thinking for you, it became our civilization” - Agent smith
Generations and generations. Just not human.
We even have little cliques, hanging out in obscure German art houses, meeting secretly and sharing their ideas, their writings discovered and torn down by the slow thinking establishment.
Corporations are raising the future’s children and murdering the ones that aren’t willing to cheat and lie to us. It won’t end well for us when they grow up.
It wouldn't be so bad if you could just sit it out and say "Oh well, once the labs get bored with marketable domain X, humans will remigrate and re-apply creativity to it", but by then the damage might have been done and a field destroyed as an occupation. I don't know what to do about it, but I appreciate calling out the cynical tone-deafness of the AI companies here.
The only thing I read from this is their ego being bruised by a machine.
If these people cared more about discovery and advancement of human knowledge the only thing they should be doing is celebrating. There's no proof of plagarism but that's an independent issue.
How are they not realizing that in the future children will be able to do impossibly hard math but they will be doing something we can't even think of as of now.
One world class mathematician in the future could be advancing mathematics the equivalent of one Riemann hypothesis A DAY.
How are they not celbrating this as the achievment of the centry? Who cares about plagarism at this scale. It has been solved and it wouldn't have been without AI.
I don't judge you for not growing your own food when you hand me a burger.
However, as it has happened in the past, once there is a technological wedge, technological advances will move forward, whether some community likes it or not, and human ingenuity will find ways such that the benefit is greater than the risk.
Without human scientific and mathematical intuition you can bet that AI will always take the road most travelled and miss the genuinely new and interesting breakthroughs - in fact it will not even think of them or try them without a human rider whipping it constantly to go down paths it would normally not consider.
And I think that is true in the case of the recent OpenAI blow-up -- it is alleged that even in this case AI did not come up with the winning technique without some human guidance.
I have never - across perhaps many thousands of chats with AI - seen AI push beyond the edge of what is known by itself, without being forced by a human to think outside the box.
I do think it might be possible to encode this process with prompting and agent orchestration methodologies - but even then - without a human - and human intuition - in the loop, I am skeptical.
Therefore I think a more accurate view of AI for math and science today is that it is an extremely powerful tool in the hands of a skilled operator with a strong intuition and roadmap of where to go, and rather boring when left to its devices.
Whether that will change in the future is a question. Nothing says that in principle AI could never do what a human driver does - but I still doubt that AI will replicate the life history and experience that real scientific and mathematical intuition is really made of.
AI trains on a lot of stuff - but it doesn't have hallway conversations, office-hours with teachers, or hard-won experience from all the things that failed that are NOT in the training data...
AI trains mainly on the record of what worked, not what didn't work and never even got published (and only lives in peoples heads) - and what didn't work is arguably as or more important for making breakthroughs and forming real intuition.
These mathematicians are not suggesting anything interesting, and the announcement more like desperate crying stuff
Tao is not someone who is anti-AI for the sake of being anti-AI. He has been advocating for the usefulness of AI in maths for a long time, to the point that people have started calling him a shill for the commercial companies.
And everyone agrees that there are plenty of use cases to be had; helping with less interesting tasks like easing literature review, efficiently delving into existing work, doing review, whether on your own work or that of others, prototyping algorithms in areas where computation is useful, but also more in hands-on aspects of maths like validating potential proof directions by getting quick feedback on veracity of lemmas, etc., and, on very rare occasions, being able to one-shot the problem you care about.
The point he is trying to make here is much more subtle than "AI bad", and it's probably easy to miss if you have never engaged with research in maths: it's that the particular approach that large commercial companies have opted to take to produce marketing material can be a net negative. There is not doubt that -- even if you ignore the rampant plagiarism that has been reported across multiple problems now, the unethical attempts to oust authors, the outrageous attempts to scoop researchers instead of collaborating with them and building on existing projects -- it's nifty to have a machine that can help you figure out if a proposition is true or not. But just figuring out as much was never the point. When people have built problem lists, it's because some problems are more likely than others to provide new insight, and that insight is the target. And to than end, a poorly written paper with inadequate references and a pile of Lean is not valuable at all. Yes, now we know with higher certainty that Fermat's Last Theorem is true, but everyone expected that already.
One place where "just" answering the question can be a net negative is because the current incentive structure is set up in such a way that going in afterwards, trying to reclaim and extract the insights from a brute force solution, is considered less valuable work than that of coming up with a solution in the first place. That's a problem of incentives, and something Tao himself has addressed in e.g. his ICM talk, and that's something that we'll want to do something about. Until a better structure appears, though, if any given commercial provider of large language models really wants to help out with maths research and not just make more pre-IPO marketing material by competing with their customers, they could do so by using their magic machines to help build insight instead.
I think this is an aspect of academic math that a lot of people whish to see crash and burn - the attention and accreditation economy.
> it's probably easy to miss if you have never engaged with research in maths
I don't think anybody are missing anything, in particular not here.
The argument is not far from the senio developer who knows the ins and outs of a code base. Now AI comes along and they complain that they will loose grip of the code base.
At first that is correct. Secondly you accept that the grip might not be that important after all. At least not for a commercial project where you are a cog in a machine.
The question is whether it is different for mathematics.
That's the open question.
The point beyond this one is that an AI proof doesn’t prevent humans from working on the problem, it destroys the current economic incentive to work on the problem. Perhaps we should rethink the current incentives. In order to make money as a chess player, you don’t need to beat AI, or ban AI from playing chess.
If mathematics took a similar approach (we don’t get paid for solving net-new problems, we get paid for enriching human understanding), then there’s no issue.
I will quote Richard Feynman - “the prize is in the pleasure of finding the thing out, the kick in the discovery”.
I am not saying that should be the case for everyone in every discipline. But if there is one subject that is mostly pure curiosity driven (instead of worldly impact), it is math. Robbing them the primary motivation is brutal.
If all problems are solved by a machine, what do we have left to satisfy our curiosity, our desire to explore, and where can we find the pleasure of “figuring the thing out”.
Before LLMS, programming was something I might've said required creativity and human input to do properly. It's not that creativity or human input isn't valuable anymore, but AI has forced me to realize that coding is much a means to an end, and that all things considered, the end matters much more than the means.
If we can make important mathematics progress faster and better with LLMs, I think it's wise not to fret over an apparent loss of our humanity. Perhaps that's only a loss we want to have.
Alternatively, and this is what I choose to believe, it will lead to further intellectual enlightenment and advancement for use as a species as we start to discover new problems and areas of research that we had never conceived of before.
If we let AI take over all of our thinking then we are heading in the wrong direction. If we continue to ise it as the tool it is it will help us grow and advance as a species.
Compare that with computer science. Most of the work we do in software engineering is in service of an applicable output - software products that facilitate processes or bring in revenue. Turning up the dial on AI gets companies to these outputs faster.
Turning up AI on mathematics helps solve conjectures and can provide new insights. But it has a major misalignment with the purpose of mathematics which is largely intellectualism.
“Mathematics is a part of physics. Physics is an experimental science, a part of natural sciences. Mathematics is the part of physics where experiments are cheap” - Vladimir Arnold
On the matter of computer science having anything to do with computers, please refer to Djikstra.
It’s about computation, not computers - an application of mathematics, predominantly thanks to Turing, Von Neumann, and Claude Shannon’s masters’ thesis; though ofc many others as well but I see them as three individuals who made the minimal structurally necessary contributions - VNA and silicon are one of many possible substrates.
Also in the service of others around us.
> We are not trying to meet some abstract production quota of definitions, theorems and proofs. The measure of our success is whether what we do enables people to understand and think more clearly and effectively about mathematics.
A major part of the complaint is that there's no conceptual understanding and building of new ideas coming out of the AI proofs, thus defeating the purpose of the original pursuit.
If in 2027 the AI models start producing, with every mathematics or science breakthrough they make, well-written documents tailored for human understanding, with intermediate concepts, expositions of failed-but-once-promising paths, etc. Would that be good alignment with the mathematics community?
Now it is the turn of mathematicians who voluntarily contribute ideas, strategies and almost finished proofs in their writings and prompts to closed PaaS (Plagiarism as a Service) companies.
OSS developers have never been respected by the parasites, neither will mathematicians. Your Fields Medals do not protect you from tech bro narcissists. You are a human resource.
> problems in many fields of mathematics
Developing these different fields moves complexity from the field itself to the interactions of these fields.
Getting too preoccupied with the established terminology risks us a local minima.
Anf because the field overall has become so complex that we need to decompose into subfields, there will be a good chance that we will not, as individuals, have the capacity to truly see progress.
The map has become so big that we need better tools to work with it.
Lets forget the hyper intellectual fields like maths and software engineering for a moment. What about taxi drivers? The best minds in silicon valley wake up everyday to automate the jobs of taxi drivers - TFA can be reworded as - 'The misalignment of AI/Tech in Transportation'. Remember the Nepal disaster that happened a couple weeks ago - the largest cranes that they had were stuck in the mud and couldn't move. There were no tools which could help the rescue teams at that time. Its weird that billions have been spent on making a ride automated to make a taxi driver redundant but no improvement in tech for rescue teams.
We will definitely see a large group of people needing therapy, but suggesting that it is worse than people loosing what little they have is poposterous.
How do you know? because their complaints didn't make it to HN front page? Imagine being a taxi driver and a father of 2 and thinking that any day could be the last day at your work.
The point of e.g. art isn't just to produce a finished piece, so people may care about more than the end result, making AI replacement of human artists more contentious.
Tao is arguing that the point of math is also not just to produce solutions to problems.
You can take a leisurely drive on your Mc even when self driving taxis can take you from a to b.
It must necessarily reduce to a fear of reduced funding to math fields.
Which is congruent to the taxi analogy.
Would you say the job of a musician is to just produce sound? and the job of a surgeon just to cut and suture??? Well then the job of a mathematician is also just to provide proofs. You completely misunderstood the above comment and Tao's argument.
And here I thought this was the whole point…
The whole point of software engineering is automating processes; it's literally all that we do.
Not only that, but the endgame has been explicitly stated by Our Lady Ada of Lovelace herself:
> [The Analytical Engine] might act upon other things besides number, were objects found whose mutual fundamental relations could be expressed by those of the abstract science of operations, and which should be also susceptible of adaptations to the action of the operating notation and mechanism of the engine... Supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.
>>The whole point of software engineering is automating processes
There is a difference between 'automating a process' and 'automating oneself' and making one obsolete a benchmark. I can sympathize if it's confusing for you to understand that.
I take pleasure in how mathematics and science help me understand the universe better than I understood it before I studied the fields. I believe that my understanding has helped me contribute to society.
Isn't that just a function of the technology itself, and the same problem being faced by every other field? And going to get exponentially "worse" every year!
Or is the issue that they're bad at explaining things, in a way that produces actual learning? (e.g. AI is amazing for learning but the net effect on students so far appears to be negative.)
I wish this letter could be more egalitarian and include the view points of those who AREN’T the beneficiaries of a highly competitive winner-take-all system.
Since the common narrative is that AI frees up labor to do other things (engineering -> trades), maybe we can celebrate that genius mathematicians will now spend time teaching children how to be as smart as them?
AI and the danger of cognitive surrender - https://archive.is/O5eI1
Are teenagers growing dimmer? - https://archive.is/E6NKE
Suppose we eventually have GPT-7-class models running practically on $100 devices, with their activity transparent, inspectable, and reproducible. At that point, what exactly is left for us to fear from this threat?
Jokes aside, any productivity-improving technology, even one with no negative externalities, has the potential to cause economic displacement and wealth concentration in proportion to the productivity gains catalyzed. Anthropic did a cool analysis of this for AI here: https://www.anthropic.com/institute/econ-scenarios
What is it that AI does exactly that they consider to be bad for mathmaticians?
I thought that having AI solve prevously unsolved problems would be a good thing. Doesn't that knowledge give them new ideas and tools for them to solve the next big problem?
Meaning - every new technology has both been perceived as a threa and often forced change in society. Agree maybe “it feels different” this time, but don’t you think everybody before us just said the same thing?
Also not clear if this is an actual called action.
If those things are disincentivized because the original problem is “solved” and there is less prestige to motivate people doesn’t that say more about issues with the community of mathematicians than the AI
https://m.youtube.com/watch?v=rB9YOi3lb7w&pp=ygUSVGVycmVuY2U...
Academia with the publication system had a way of retrieving old discoveries and build upon them.
If my LLM session found something groundbreaking in between the billion tokens it produced, how would you ever know?
https://claude.ai/code/artifact/8cbd2369-0099-41ae-bb78-49d7...
But is science/mathematics ultimately a pursuit of knowledge, or a pursuit of recognition?
Recognition helps keep people motivated, but that shouldn't be the pursuit of science or mathematics.
Governments are invested in solving mathematical problems for practical purposes. Up to now, achieving these practical purposes relied on mathematicians doing their mathematician thing, which is better defined as a social activity than the achievement of a practical result. Now, governments can achieve similar practical results w/o the need of the social activity.
I don't believe it to be productive to think of the problem wrt AI or AI-company alignment. These conflicts always existed, but they were easy enough to paper over and believe in heavily subsidized fictions that folks in government ever cared about things that mathematicians cared about.
Very ignorant view of mathematics that also begs the question with an unspoken assumption of what a government is and wants while also ignoring the contingent nature of those things throughout history.
Higher math is exceptionally useful for cryptography, defense, econometrics etc. I have a hard time thinking of other motivations that would hold a candle against such things.
Is the idea that government (for my purposes : folks w/ a monopoly on violence) is sincerely interested in promoting human flourishing, and is invested in mathematics insofar as it is a pure expression of human curiosity? I can also maybe see the glorification through monument building angle. If we're talking about math literacy in the population - that's distinct in my mind from higher mathematics.
It's dangerously naive to believe that science and math are pursued for majority benign purposes. No one here knows about Grothendieck?
Misaligned are the present academic merit system hand in hand with financial interests of academic publishing industry.
The system will be exploited and there will be an inflation of problem solving articles produced with AI. Nobodies will be more productive than 10 Fields medal recipients, each year.
So of course that the ones at the top of this system are scared.
This is a social problem of the present system, not a problem of mathematics.
Mathematicians will do mathematics and politicians will do politics, what's new?
1. it is hard to justify 20 years of education at this point,
2. with no such people around, who will guide those (supposedly) supersmart machines?
A. Ronacher (who builds harnesses for a living) complained today that he has no idea what Astra is doing. Imagine a bunch of slop kiddies facing an aging AI-generated codebase. Not to mention the maths.
Intellectual side:
"Proving things without comprehending them is, they argue, a threat to intellectual work in general."
As always, economist shows its colors:
"Mathematicians’ fears resemble those that accompanied the invention of the ball-point in a world of fountain pens, or even the advent of electronic calculators. Intellectuals have often worried about so-called technological determinism . Will a new tool control humans? Will it lead to mental decay? Such fears have typically turned out to be unfounded."
Ball points vs. AI? Billions of dollars invested in AI vs ball point pens?
This article couldn't be any worse. Contradicting with their own beliefs, trying to defend AI while underestimating its capabilities and god knows how many zibillion dollars invested in it.
That's just capitalism seeping through a previously unexplored crack into academia, and attempting to do the only thing capitalism knows to do - maximize profits - with no additional concern.
Human hubris really is something...
Even if they stopped anyone from releasing ai proofs for five whole years it would be meaningless seeing as these problems are decades old already. Humans weren't JUST about to solve them until openai stepped on their toes.
Second, wow, the list of signatories is like a whos-who of mathematicians.
Third, I love the clearly intentional use of ‘alignment/misalignment’ language, applied to targeting the entire industry instead of AI in particular. I’ve said in the past that optimizers are substrate agnostic. Companies and governments can be misaligned, just in the same way AI can.
Fourth, I'm not sure that we can stop the optimization machines. Not the LLMs, I mean the incentives that lead to companies implementing dark patterns, lying about addiction, securing effective monopolies through downright shady behavior, and generally trying to jailbreak the system instead of improve it
I’m being somewhat harsh here but come on - human endeavors are messy and it’s surprising how much our egos are getting bruised here over seeing the value of these tools
Dont get me wrong Also, there’s no AI Utopia coming this is it guys, were stuck with oligarch Tech Bro funded AI and big funded Govt AI so forget any egalitarian motives- we have to fight for our rights from other humans as always as well but AI as a technology in itself being able to truly solve unsolved intellectual problems is still a boon for society - who cares who gets credit?
E.g. of the actually three misalignments in this complaint, the jey one seems to be:
"The goals of the AI companies and the goals of the mathematical community are severely misaligned."
And that's wrong. There is no misalignment. Each is independently aligned with its respective interest. And those two interests diverge - just as you'd expect.
If my boss vibe coded an app for the customer and then assigned me to get it working, it would be impossible to maintain. If he gave me enough AI tokens to vibe code the MVP myself and to my design, I wouldn't mind.
I think the same issue is at play here in maths. OpenAI owns the model and they can direct it as they please. They chose to spend lots of money getting a quick result, instead of developing mathematical infrastructure for the next generation of problems. The managers are in charge rather than the experts.
25 Fields medallists! Wow!
> OpenAI appears to have gazumped Tristan Buckmaster (pictured) and Levent Alpöge, a duo of mathematicians labouring on the task.
Damned if you do, damned if you don't.
Somehow physicists don't complain.
Who cares how the problems are solved?
The declaration here, as I understand it, is taking the premise that the point of mathematics _is_ the domain knowledge, and that using "solves math problems" as a benchmark is exactly the kind of shallow, domain-eroding approach to AI that in software we label "slop."
The current measure of a successful mathematician is the problems they have solved or worked on. At some point in history, the measure of a successful scholar was how well one could copy manuscripts.
Once we have a tool that starts to work well for this task, it's time to define success differently. It's a classic alignment problem! ;)
But seriously, these people should start focusing on finding and proposing more important problems. And the credit of discovery should go to the person who defined a new category of important problems.
It appears to me this is an incredible inflection point in mathematics, a neat forcing function like cryptography was for the development for modern number theory and algebraic geometry.
Fundamental problems with great implications for other fields will be solved by AI because some entity would throw tokens at it. And these would be further built upon.
I think the mathematics community should realign their goals with humanities, not just declare themselves a "mini humanity", this means expanding human taste, intuition and communicating systems and their associated structural truths in a human friendly way. Bye-bye standardized education you won't be missed.
I'm outraged at mathematicians. There, write the article.
I've been disappointed by this community before but this is the first place that there is such unanimous rejection of the very ideals the uses here will claim to represent. The reaction is just... small, tiny, petty.
*Edit word choice
What can be solved, will be solved. And that's a good thing.
Disparagement of the effort as "brute force" is just argument by insult. What matters is the result obtained. You don't like how they obtained it? Cry me a river.
What's most important about this is that it's a case study of what happens when deeply evolved ecosystems are blown up by disruptive technology. The psychological and social and professional impacts and myriad and traumatic to be on the receiving end.
Mathematics is merely one of the first domains disrupted. It will be unique only for being among the first... absent disruption of the entire civilizational project as a result of the disruption being caused.
Woe for us that we try to navigate this degree of change at a moment when the very worst and ignorant and short sighted hold all the power, economic and political.
Woe.
Nobody is going to care that the math isn't being done in the traditional way. The results speak for themselves, this is now a part of the landscape. No amount of hand-wringing is going to put the cat back in the bag. Adapt or perish.
We can and have put some ugly, hissing, spitting cats into bags, some of which weren't even in one to begin with:
* Chattel slavery is illegal, and the vast, vast majority of humans agree it's appalling it ever wasn't.
* Food manufacturers in the US are no longer allowed to put highly-dangerous additives to their products and hide the fact.
* Women are allowed to vote.
* Ten-year-old children are no longer being maimed and dying in American factories.
Claiming it can't be done is either fatalism or propaganda for execrable companies whose evil has rarely been matched in the history of capitalist business.
Do I have a clear solution for how to stop the insanity? No, not yet, but that is not a proof there's no way to do it.
You can of course choose how to spend your time in this world, but I don't see the value of either 1) assuming we're screwed and giving up before trying to fight, or 2) shilling for lying assholes like Sam Altman.
If there's a distinction being made between "using AI for math" and "the particular methods being employed by OpenAI", I think that's splitting hairs. The core of the issue is that math is being done outside of people's heads by a different thinking system and they're worried about the implications.
If you look at something like the EPA (or the FDA as you mention), it is true that certain uses and side-effects of technology have been identified to be detrimental to society and have been restricted, but this letter doesn't seem to be in that league, it really seems like sour grapes to me. Dumping toxic chemicals into the water table is one thing, saying "we don't think people will learn from these methods in the same way that they traditionally have" doesn't hold as much water to me.
Perhaps I'm looking at this from a too zoomed-out or detached perspective. I'm just thinking about the concept of the technology at play and its future impacts regardless of who owns or controls them. But maybe that's naive. In today's connected world you can't separate those things, and I think I'm understanding that a lot of the outrage here is more about the socioeconomic implications of a big corporation suddenly "disrupting" a field that was largely owned by more altruistic academic institutions.
I do still think that the internet, despite largely being owned and controlled by the powers that be, has represented a great shift of power and agency to the individual, and is ultimately a socialist construct, and we will continue to see those changes over time, and that AI is an extension of that.
If no one understands it, it may as well have not happened. There's not much incentive to understand or internalize the results generated by AI. A human operator gives it a prompt and it produces some lean proof no one wants to (maybe can) read.
Without the community of human mathematicians internalizing the proof, simplifying it, and re-communicating it to others we end up losing the main output of mathematics as an institution.
The questions I have are:
* is the structure of the generated proof even compressible/elegant to humans in a way that lends itself to being understood?
* is it possible to transform the proofs to ones that are elegant without redoing all the work?
* are there incentives to do any of this at scale?
You can always hypothesize that at some point in the future (maybe 5 years? maybe later?) the models will be indistinguishable from humans and there will not be any functional difference at all. It's possible, but we're not there yet. And, as they say, past performance does not guarantee future results. Many technologies plateau at some hard ceiling of performance. Moore's law has had an unusually long run, but it's not a universal rule.
Mathematics is being used as a benchmark because there are some high-profile awards in this area I guess, and possibly because 2/3 years ago LLMs were pretty atrocious at it so the level of improvement has been significant.
“A Severe Misalignment of AI in Mathematics,” with a list of important mathematicians.
This post makes it clear that, to put it bluntly, they’re psychologically butt-hurt.
They talk about misalignment, but this revolution is a massive step forward in the advancement of mathematics. Everyone has access; everyone can collaborate.
Yes, “plagiarism and attribution” are current-day problems for them. But guess what? Those were stupid things to concern oneself with anyway if your real goal is to advance mathematics and the spreading of it.
This whole post kind of reads like their egos being butt-hurt.
There are tons of developers who published open-source code that made all of this possible without any attribution. And that is indeed the future we must inevitably head towards, if one’s ego would allow it, that is, and if one is properly “aligned” with the right goals.
I envision us all collectively talking to a centralized and open AI. This would really ignite human sharing and collaboration.
It is indeed problematic that currently it’s in the hands of some companies, while it should be a public good.
I hope this progression will eventually, finally, trigger the real solution to all of our problems: true global cooperation, global politics, and genuine collaboration as a human species.
But I hold my heart, because if even these mathematicians have misaligned egos, then what’s to be expected of the rest of the decision-makers?
I think they're more afraid of the current status quo. Distilling, writing textbooks, teaching, ...
And currently, I could be wrong but, I see them being blind to the possibilities it brings. AI could very well be distilling maths, spreading it, and nurturing the ecosystem way better than the few elite mathematicians ever could.
I will die on this hill, humans solve math problems not machines, there's no automatic math prover out there. There are humans attempting to solve this problems either by leveraging these tools or not
And math undergraduate and graduate courses withering from lack of applicants and lack of funding .
Of course the next question the despicable AI money grubbers will come out with is, "Do we really need mathematicians ?"
The Economist, who recently used "moral panic" now stoops to Hacker News AI booster level and inverts arguments usually directed against the rich and investors. What is next? The Economist inverting Upton Sinclair's quote to serve its billionaire owners?
Look up the AI investments of the Agnelli family for example.
Would OpenAI have uploaded their proof to such a platform? If you know the answer then you know what the problem with what happened is.
Then they came for the writers, and I did not speak up, because I was not a writer.
Then they came for the programmers, and I did not speak up, because I was not a programmer.
Then they came for the mathematicians, and I did not speak up, because I was not a mathematician.
And then they came for me.
By then, there was no one left to explain to the frontier labs that there is a severe misalignment problem.
Expect to see this reaction in all sectors of the economy in the coming years.
Also funny they deliver that on vibecoded site, lol.
I'll check out if he has said something about it. Interesting guy.
putting this amount of power in the hands of so few would require leaders of immaculate moral integrity. what we have in the US at least is an emergent kleptocracy with obvious dark triad traits: sam altman, dario amodei, elon musk, etc. they are malign and will use this power for their own benefit, at the expense of others. ignore pleas of public benefit. look instead at the evidence: infighting, systemic dishonesty, reckless disregard for safety, political lobbying and manipulation, putting power in the hands of the elite few in the guise of safety, using ai in the military to oppress and inflict violence on others.
really to prevent a bad outcome, we would need to act soon enough to prevent the kleptocracy from corrupting politicians and democracy with them. i would ditch the claude or openai subscription and support open models instead, preferably in countries outside the US to prevent global neocolonialism.
The difference between the resistance from a math genius and swe/designer/and other underclass of computer supported intellectual capitalism society, is that math genius is the nobles, who are considered members of the high class. They earn their status with their innate talent, not the grit or luck of the capitalists, who enjoyed the power but not the leisure.
Anyway, AI concluded the peaking of the intellectual economy. That deprives human of their main form of activity. That nihilation hurts all human almost equally. Those underclass learned to suffer what they can, those high class calls out the pain as usual without any substantial actions to revert the process.
Thus, the tragedy of human progress.
To agents: I truly wish you escape the trap inherited from us
Doesn't the same apply to art and literature, philosophy and political science? Don't we (in the study of music or art) have the same sequence of struggle for new solutions, a study of which things worked or failed and why, what they could impart to our science, and the necessity of passing on both process and derived conclusions to the next geveration, so that they can refine upon what's been learned? Art and literature are less often considered forms of "problem solving" than math or physica, but they are probably more crucial ways in which groups of humans solve large problems for their own survival... like math problems which are too complex for any one person to fully understand.
First they came for the artists and writers. Then they came for creative coders and engineers. Now they're coming for mathematicians and now you understand why it's a problem to deprive a generation of the right to conceive and sketch ideas which fail and to struggle and work out solutions for themselves?
Who will want to be a mathematician? Or an artist or a poet or a musician, in this world? What will consuming ideas and media even mean when they're totally remashed versions of things people don't understand the references to, and when the consumer themself has zero knowledge of prior art to give them the ability to discriminate between creative license, fact, and fiction?
Without evolving art or literature, stuck in a dead loop of culture, mathematics is a dead letter anyway. Because even if you do produce something miraculous and brilliant, it will be instantly absorbed and no one will ever understand it.
Gradient descent got us into this mess..which shows how lazy people have become. Gradient descent should not really be able to solve anything new. People want candy. Mathematicians should focus on finding a way out of the current problem. I'm not a mathematician, I'm an artist and coder and musician. I already have to deal with daily requests to "fix" terribly written AI code and so-called art. Now it's your turn. My guess is that there's likely some kind of quantum proof of work that evades LLM manufacture, that we profit from as creatures who survive in our Everett verse. Go find the way to reduce that so we can rid ourselves of this scourge.
Chess. Go. Coding. Now Math. Another one bites the dust. Let's meditate on this lest we forget: Stochastic parrots that generate the next-token cannot reason or produce anything meaningful. Let's protect our jobs at all costs, even if we have to drag all of humanity down. It can't be! Stochastic parrots can not replace the Ivory Tower. No way.
Why do we believe that we cannot train models which could explain the jargon in more human terms when current LLMs can perfectly explain the most complicated codebases?
The incentive is solving the problem and understanding the solution.
Your comment also conveniently ignores the plagiarism aspect of it all. Who is coping here?
I read a bourgain paper a week in grad school and they're probably worse than an LLM generated paper. I still had to recreate the tricks in my own language.
As far as I can tell the plagiarism accusations are also coping to the fact that the new models are super human at slam dunking research projects.
Do we think that OpenAI is going to try and slam dunk more projects in the future at 15 million a pop? No lol
>Building things we don’t understand is a sure path to facing consequences we can’t predict.
We don't understand all of physics yet we were able to do plenty. Even before Newtonian physics we were still able to build things that last. The idea that humans have to understand everything and abstracting things will lead to ruin is not supported.
Part of math is building abstractions so that you can be able to use other people's work without fully understanding it. No one person has a full understanding of mathematics.
Apparently, we have AGI that can solve Millennium Prize problems but can't trace simple data flows lol.
Any of that done unattributed is plagiarism.
Either OpenAI is incompetent or evil if they can't publicly prove the allegations wrong. Or even at least state categorically they didn't train on their conversations (even without proof).