> However, it is now the case that AI can output mathematical arguments in situations without the human who prompted it being able to understand the arguments, verify them, or take responsibility for them.

I read this as "anybody can prompt, few can understand". And "we need more who can understand". If we had more mathematicians (than we have today) all of them piloting advanced models, the pie would grow. The problem is that AI capabilities drain (by disincentivizing) the education pipeline that would get us those mathematicians, and if recent rumbles about what AI is doing to education are to be believed, it does so many years before students even get to grad school.

IMHO, it is not that bad. Not having any human who understands linear algebra after the Butlerian Jihad is a win :-) .