What's a sample prompt that you've used? Every time I've tried to use one for programming, they invent APIs that don't exist (but sound like they might) or fail to produce something that does what it says it does.
I had a funny one a while back (granted this was probably ChatGPT 3.5) where I was trying to figure out what payload would get AWS CloudFormation to fix an authentication problem between 2 services and ChatGPT confidently proposed adding some OAuth querystring parameters to the AWS API endpoint.
It tends to fall apart on bigger asks with larger context. Breaking your task into discrete subtasks works well.
It's like infinitely tailored blog posts, for me at least.