When it gets to questions like these I feel that we transcend discussions of technology and end up on questions of philosiphy which aren't going to be going anywhere anytime quick.

I also feel that AI should be used to augment not replace human decision making, it seems that where AI shines is problems that are well defined with well defined solutions, and because the AI doesn't get tired, hungry or distracted it can do that really well, but it fails in novel situations[0]. As such it seems to me our best bet is to have the AI provide suggestions rather than have complete control.

0. What is meant by that is a read an article, can't find it now, about using AI to diagnose breast cancer, what they found is that about 90% of the time the AI could accurately check for breast cancer, but the other 10% of the time was an unusual mammogram or something relatively rare, and in those situations the AI would often misdiagnoise.