problem is most of the code chatgpt spouts is wrong, in so many subtle ways, that sometimes you just have to run it to prove it.
so basically you have to be better than chatgpt at that particular task to spot its mistakes.
using it blindly it's similar to the Gell-Mann amnesia effect
https://theportal.wiki/wiki/The_Gell-Mann_Amnesia_Effect
said by someone who uses chatgpt extensively, it is good for the structure, to get an idea, but as a code generator it kinda sucks.
I am not a programmer and only know some very rudimentary HTML and Java. After hearing everyone enthuse about how they use ChatGPT for everything, I thought that I could use it to generate a page that I thought sounded simple enough. Gist of it was that I needed 100 boxes of the same dimensions that text could be inputted into. I figured that it'd be faster with AI than with me setting up an excel sheet that others would have to access.
Instead, the AI kept spitting out slightly-off code, and no matter how much reiterations I did it did not improve. Had I known the programming language, I would have known what needed to be changed. I think that a lot of highly experienced people are using it as a short-hand to get started, and a lot of inexperienced people are going to use it to produce a lot of shoddy crap. Anyone relying on ChatGPT that doesn't already know what they're doing is setting themselves up for failure.
Interestingly, the same applies to text-to-image programs. Once you used these for a while, you realize their utility and value are little more than an inspiration or a starter. Even if you wanted to ignore the ethical implications, very little they produce is useable. LLM are amazing. However, their end-product application is overrated.