Humans as a group has proven to be able to build everything we have today. No AI has proven to do anything similar, there just isn't much data there to make us confident in their abilities thus far.
Conversely though, how many people are killed in the world every single day simply because of human error?
When (in the US) a 16 year old gets their license, we don't ask them to provide a formal proof of their driving technique that shows it's impossible for them to ever get in a crash. We say 'you've had the training, you've demonstrated that you can safely drive, be careful out there'.
We have a good understanding of this risk and have collectively chosen to accept it (while many countries have chosen not to accept it in the case of 16-year-olds.)