I've already worked with way too many engineers who copy/paste code without understanding what it does.
The junior devs, lacking the experience that provides the context or knowledge to analyze the "shape", let all manner of minor and major issues from the generated code slip past.
I really notice this particularly with say less popular frameworks or code libraries, where many LLM code generators appear much more likely to hallucinate invalid code. I see similar issues with DSLs too, where the LLM starts to just guess at the DSL syntax. I really hate when an engineer new to a framework just goes to the LLM first, without at least spending a little time learning the framework for the task.
All said, I suspect these code generation tools will get good enough that the rate of errors goes down enough it maybe won't matter in the longer term.
It may turn out that ChatGPT results in a new generation of developers who produce terrible code, full of flaws, with no understanding of how it actually works.
Or... it might be that ChatGPT helps newcomers get over that horrendous initial learning curve, start by writing bad code... and then get better at it.
I'm currently cautiously optimistic that the latter scenario is going to win out.
ChatGPT could also empower individuals who may not have considered learning programming to create practical programs that enhance their daily routines, similar to how VB6 significantly reduced the barrier to entry for developing various applications.
I don´t rely on ChatGPT to write any code for me but i do use it whenever there´s a concept i don´t understand and if there´s any problem blocking me from getting forward in my many programming books. Having a tireless teacher/mentor with extensive knowledge and who does not care however stupid question you ask is a very good tool for learning.
I'm not sure GPT helps with any of those yet, as it seems to be more likely to spit out incorrect-to-sortof-okayish so far.
"Um, I'll tell you the problem with the scientific power that you're, that you're using here. It didn't require any discipline to attain it. Ya know, you read what others had done, and you, and you took took the next step. You didn't earn the knowledge for yourselves, so you don't take any responsibility for it. You stood on the shoulders of geniuses, uh, to accomplish something as fast as you could..." (Emphasis mine).
I was using it for debugging WINE - where I have no DirectDraw4 / DirectX6 experience, and it's much nicer than trying to trawl through ancient MS documentation myself.
This is called reflexion and has been found to be an effective way to improve response accuracy and usefulness (also reduces hallucinations).