In the case of ChatGPT, you can be almost sure that if a non-trivial amount of code works in production then it's an unlicensed paraphrase of some human written code. It treats code (and everything else) like prose, hence the atrocious results in the original story.

That does not mean other coding-optimized AI models won't be able to do much more, symbolic reasoning about the data, maintaining consistent variable and function names, understand libraries, respecting language constraints and invariants etc.