What happens when the AI can write itself? What happens when it can design its own hardware? And assemble it?
What have we created when it is fully capable of self replication?
I remember when Garry Kasparov insisted that centaurs -- computer+human — do better than just a computer or just a human. He held out longer than most in this belief. But seriously… can you name a field where computers and bots can NEVER beat humans?
I don't disagree whether computers are good at computation. That's what they're for. I disagree that AI can replace experts because expertise is a moving target.
Where AI approximates a response given at some point in time or within a range of time written by a crowd of people who are not necessarily experts, experts must inevitably contradict themselves on something and update their theories. AI as currently implemented will always lag behind human experts.
In other words, do you seriously believe that an AI trained on scientific publications from the middle ages would eventually rediscover what we know now and beyond? I think it would be stuck forever hallucinating recipes to turn lead into gold.