Yes, people can make plausible crap much faster now.
From my experience, since AI tools has been adopted (unofficially) by our developers, the amount of bugs increased dramatically. Feature delivery is much slower due to multiple rounds of testing after fixes.
With AI, to realize the productivity benefits, you just have to adopt a process that create new stories for all those tasks. Then you'll start to see massive improvements when you look at your close rates. Just think of those new bugs you can close now that you're using AI!
Instead it will be just as it is has been for years, let’s send this (in this case) AI generated data to Malaysia or Bangladesh and have them clean it up and then bill it as perfect AI solution.
These people in poorer nations have been cleaning data for ML and AI for years. It’s just adding another loop back in.