My experience is that they spit out reasonably looking solutions but then they don't even parse/compile.
They are OK to create small spinets of code and completion.
Anything past that they suck.
It's actually hilarious that AI "solved" bullshiting and and artistic fields much better and faster than say reasoning fields like math or programming.
It's the supreme irony. Even 5 years ago the status quo was saying artistic fields were completely safe from the AI apocalypse.