Imagine if humans couldn't understand multivariable calculus, but we had access to an AI system that developed it, it initially seemed useless, then another AI system found a predictive model of electromagnetism using it.
When you have full AGI of course you no longer need humans to understand math.
> Imagine if humans couldn't understand multivariable calculus, but we had access to an AI system that developed it
Developing multivariable calculus requires much more than just solving problems though, it requires defining an entirely new system and space. That is not the situation mathematicians face today, modern AI cannot do that.
When talking about mathematicians and AI don't use fictive examples, we can look at what AI can do today and extrapolate that they can do more of that tomorrow, that is what we have to work with.
In the case you posit where AGI exists there is no reason to even discuss what is left for humans to do, since AGI is defined as when humans are no longer needed for anything, the AGI can do every bit of thinking humans can.
Anyway, if we put the AGI framing aside, I think the main point you're making is that AI mathematics hasn't yet demonstrated the ability to theory-build in the way that the great human mathematicians have (Grothendieck, Scholze, etc.). And I'd agree with you on that. Where we disagree, I suppose, is I think that capability is coming -- I don't see anything that would prevent its development.
The idea of AI stepping from a graph theory/combinatorics innovation to some new and useful algorithm isn't crazy.
If I understand you correctly, you're just qualifying that that will only be the case when AGI exists. To be clear, I actually disagree with you here because I think it's very plausible to find a use case for human-incomprehensible math proofs before AGI exists. I'm just saying it sounds like you're agreeing with the parent comment that math is not purely about human comprehension.