138 karma · joined August 16, 2024
The "if" is the problem. If it happens, then of course, let AI do it. For the moment AI is still bad at those type of tasks [1], so the discussion shouldn't focus on highly conjectural situations. We can't destroy the scientific ecosystem based on vague speculations.
[1] There are real reasons: it is not obvious how to optimize an LLM for doing basic science or other ill defined tasks. On the contrary, optimizing for writing a proof that passes the Lean test or code that passes the tests is a different story.
This is beyond naive.
It's not obvious at all that it's possible to train it to operate independently. So far, AI has been succesfuly trained only for highly controlled and verifiable tasks (code generation and mathematics).
We already had that before AI. The problem was precisely that it's too inefficient to make a new system for each different task. Of course, the new AI is more multimodal, etc, but up to what point? Not clear so far.
Of course this doesn't mean we may need less amount of humans for certain jobs, or that I'm the future AI will gain new abilities.
I didn't expect them to throw millions of dollars at each famous math problem. But one year ago we already had LLMs that solved IMO problems, no?
> Are you sure? (The numbers I've heard, which I admittedly have no very strong reason to trust, don't seem that way to me.)
Math has very little founding compared to other science domains. Also, if you filter mathematicians by specialization in PDE and that have worked on Navier-Stokes, then you end up with a very niche community.
> For instance, suppose you give one of today's frontier models some of those chain-of-cubes rotation puzzles. How well will it do?
I feel like this is not the correct way of thinking about it. We can also ask, for instance, how well a state-of-the-art algorithm for the salesman problem works on a particular graph topology. People do PhD thesis on topics like that, so the answer is not obvious at all. For LLMs we still don't have a curated theory that explains what they're good/bad at, and that you don't see how to extract an answer from the definitions is no surprise since this is obviously not an easy problem. But all this is normal because this is a rather new topic (models of this scale appeared when? 3 years ago? That's nothing for science).
Anthropomorphizing LLMs has added so much noise to this discussion.
There are many automatic theorem provers that do very clever stuff, just as the underlying theroy describes.
> I am shocked how people can deny that solving Navier Stokes requires some sort of intelligence.
It is absurd to waste time discussing whether it is inteligent or not. It is just an algorithm, we know how it works, and it does exactly what we expect it to do. LLMs are not magical things. The main difference is the scale: for Navier-Stokes they spent in 3 days more money that the whole mathematical community over the last 20 years easily.
By the way, I'm not saying that LLM's are useless, that I'm anti-AI or anything like that.
Let's be honest: we don't know. Maybe you're right, but for the moment it's more likely that you're not. And countries cannot bet on that vague intuition at the cost of destroying their research communities and world leadership (which takes decades if not a century to achieve).
I don't think we're discussing pedagogy. Good _research_ exposition is instead related to communicate your intuition and way of seeing things. The conceptualization of a given situation or problem is what is valuable, how you connect it with other stuff, etc. It is then up to you to memorize and interiorize it.
It's not about moving goalposts. What you're not understanding is that, even if for you is crystal clear that AI will be 100x smarter tomorrow, those in charge cannot simply bet all on that. Right now AI cannot really replace the core sauce of mathematicians (all the "understanting" and "asking the real questions" stuff), so it'd be unwise for, say, countries to start making decisions as if AI is capable of that.
It should be noted that there was a manuscript, available online since the beginning of 2025, with a solution to the Jacobian conjecture:
"Adrian Vasiu claims that the 7 page AI paper on the 3D Jacobian conjecture counterexample used notation and concepts from a draft of a paper jointly written with Alexander Borisov and Ofer Gabber, dated to January 14, 2025 and made publicly available on January 16, 2025."
The extract is from wikipedia, where the sources are given.