In my experiments with LLM's one of the most obvious issues is project structure and organization, particularly in the context of future objectives. It is way too easy to have an LLM go in circles when analyzing ideas for project structures. I have seen them propose layouts that, while workable, are far from ideal.
The same applies to code. It can write usable code faster than a human. However, and, again, in my experience, you are constantly refactoring the results.
In all, this is a new tool and, yes, it is useful. It's amazing to me considering the idea that the whole thing is based on guessing the next token. It doesn't really understand anything, yet, you'd be hard-pressed to prove this to those who don't know how we got here. I've had a few of those conversations with non-coders using ChatGPT for a range of tasks. It's magic.