Human beings struggle to communicate system requirements to other human beings. We're supposed to be "smart", communication is supposed to be something we're good at.

Getting requirements to an AI is going to be grotesquely challenging, and even when we're able to get that far, testing the work is going to be an ever greater challenge.

"Exactly what I asked for, but not what I wanted" is going to rise to an entirely new level.