What you are saying is akin to saying that humans can naturally metabolize methanol so we can employ humans as industrial formic acid producers.
There is a difference between what's easy/hard for a human vs computer, and LLMs haven't changed that. You might expect a computer to be good at tasks requiring prodigious memory and compute, and it turns out that some of these long-standing math problems are of that nature - not requiring new breakthroughs but rather just massive exploration of what is already known and what they were trained on.
There will no doubt be more math results like this, but presumably also ones that are "hard for a human, easy for a computer", requiring massive search (e.g. find an example/counter-example cf Navier-Stokes & Jacobian conjecture) rather than creativity.
Look at what's happening in the math community- rather than excitedly embracing the power of AI's ability to generate new proofs, they are screaming for it to slow down (and quite a few want it simply stopped). Many scientists, even in ostensibly more real-world fields are broadly cut from the same cloth.
A much more practical reason we're not yet seeing a lot of headline scientific mathematical breakthroughs is just that it is ungodly expensive! e.g. The Navier-Stokes result cost around $20M at API prices, and academics just don't have that kind of money to spend. You'll see more mathematical and scientific results from practitioners when either the cost of compute needed for these sort of brute force results is more in line with the size of academic grants, and/or the AI companies donate more compute to the scientific community.
There are different reactions from different mathematicians of course - Terrance Tao vs Cedric Villani, and no doubt a lot of shock at the speed of advance, but it seems the reasoned complaint why they don't want the AI companies themselves working on these problems is because the outcome is not the same - you get a result that in of itself may have been suspected or useless (Navier Stokes), but no write up of any new math or insights that were developed along the way, which is the real reason mathematics and people like Erdos pushed these famous problems in the first place - because they were expected to yield interesting mathematics, just as years of work on FLT had done. Imagine if instead of Wiles's work, and all that had gone before him, all we had was a $20M compute bill, hundreds of pages of impenetrable math, and a billion lines of Lean proving it was true?!