What is their pay going to be justified by once computers start conjecturing and proving theorems on their own?
Otherwise they’ll be the ones like Erdős who pose the questions in the first place.
Either way it will always be humans who decide what matters. AI is speaking our languages, not the other way around. We’re in charge. It’s impossible for us not to be, unless we can train an AI from dolphin data or other natural phenomenon.
The AIs intelligence is tuned to us and in 300 years we’ll need new training runs for the update from human zeitgeist language and the 2200 century famous mathematicians.
AI companies are accruing power by virtue of its knowledge and ability to do work. If endowed with agency, which seems likely at this rate, it is the AI itself that will be powerful. And we'll be in charge because AI is trained on human language? I can't fathom the logic behind this.
Put differently: There is no natural law of the universe of why we pay people to do work. It just works well for us currently. If it stops making sense we do something else that works well for that new currently.
There is real risk in doing it too early as well. Imagine if we thought the same ~50 years ago when microcomputers started to appear. If the tech companies had to pay the salaries of every displaced job we would have ended up with severely hampered progress on computers and then a job shortage as we realized there were just different jobs now, not necessarily no work quite yet.