It's almost useless for high-level design and architecture. Those tasks do take more time than writing code, but they're super-charged by rapid prototyping, which ChatGPT excels at. For example, right now, if I'm picking between three libraries, I can have working versions of my code implemented in all three in a few minutes, and built out a bigger mockup in a few hours. I can make a more educated decision from there. In the olden days, that'd be a few hours and many days, respectively.
It also does really nicely for code reviews, which results in better code. That also speeds up maintenance and, later, debugging. That's especially true in domains I'm less familiar with.
Some maintenance, it excels at. I'm the author of a major piece of open source infrastructure, which is widely used, well-architected, but every technology used is obsolete (JQuery, older Python web frameworks, legacy database, etc.). I feel like I could rewrite the whole thing in React+Node+etc. in a few weeks. There is close to zero architecture work -- it's mostly having GPT write -- or sometimes just translate -- code piece-by-piece.
I don't feel like GPT-4 obsoletes software engineers as it's used yet, but it does really raise the bar on what's possible to build.
Of course I'll try it and see how it does. But it struggled and failed to produce what I would consider trivial code when I tried to create a single usable page to spec in react native, so I'm not terribly optimistic.
What's interesting is that GPT isn't trained to produce high-quality code. It's trained as an autocompletion tool. I'd be curious how smart such a system could be if we knew how to engineer it for quality.
It won't find subtle race conditions, abstraction violations, or deeper issues.
I find it makes code reviews shorter and more productive, so perhaps you'll spend 10% of your time instead of 60%. You won't spend 0% of your time, though.
I just hate how slow it is even with Plus. For analyzing code for debugging you’d ideally have something that moves a little faster.
I guess we’re in the 56k days of LLM watching it print line by line.
GPT is just Spreadsheets in a way. Lot of companies had business users and SMEs create contraptions based on Excel and then when it's a hairy mess or a rube goldberg machine, there are "digital transformation" projects that are done to make them proper apps.
One of the examples from GPT wasn't even responsive and didn't work on mobile.
But...some folks are creating iOS games without any Swift experience, so that's cool.
"I'm getting this error <error> with the following code. Please add log statements that will test for all the things that could be going wrong <code>"
Saved me about 15 minutes yesterday. Not only does chatGPT generate the probably causes of the error, it knows how to test the code for those errors. The human doesn't even have to read the error message anymore.
We need to find a way for the system to iterate, debug, write unit tests, and repeat the loop. In theory it's doable, but we are yet to see the system which does this.