New theories and insights are typically created while working out proofs. If proofs now suddenly fall out of the sky (cause LLMs create them) then that work is not done which means the substrate on which new theories and questions and conjectures used to be grown disappears. It's in that sense that the math community (and thereby society as a whole) will lose something.
It's similar to how software engineering will need to find a solution to train their next generation. Current generations have all been through manual steps of designing things from scratch and writing them by hand. That's what allows your 10x engineers to understand whether what their LLM tools are doing is good and how to massage those tools to do the right thing. A junior engineer who has only ever used LLMs to write code and create architectures does not just not have that experience but also won't acquire it. You can't just say "we don't pay them to have fun and learn, we pay them to produce results". In the short term that is the case, but in the long term you as a company and we as a community will lose out.
I'm not saying don't use AI tooling. I'm saying that this is a hard problem which we yet to have to find solutions and approaches to. As a software community as well as as society in general.