Is there any evidence that the Waymo approach involves "the AI" at all? I think you're projecting the approach of other, failed self-driving efforts onto Waymo.
Waymo uses ML/AI extensively. Here's a blog post from earlier this year about how they're forgoing CNNs for a "hierarchical graph neural network." https://blog.waymo.com/2020/05/vectornet.html
Is the concern that these nets might be trained in such as way as to infer the wrong thing in cities other than Phoenix?
I doubt they'd blindly deploy in another city with a network trained entirely in Phoenix. That's not something even an amateur MLE would do.
Yes, overfitting to the training set is a problem with AI.
Are you suggesting that Waymo might not use machine learning? What do you imagine they do instead?
No, I understand they use NNs for object classification and so forth, but compare their approach to Tesla, where the overall architecture is camera → a miracle → steering actuators. It seems from the outside that Waymo relies less on the black box.
Also rule based systems / manual decision trees can overfit :)