Death by semantic error.
An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.
An AI tried to tell me the {} opens a new block scope in PHP and any variables in it are scoped to that block. I nearly lol’d and the code it gave me was so wrong it was cringey. Hopefully it is better at Python.
Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews).
Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I cannot say it’s bad at doing it; it just is not necessarily sound or even working code sure, but that’s the same as with most human first tries. Then you iterate and make better. My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.
"good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc"
versus
"good at finding solutions to programming problems."
GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words.
All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.
I do! I use code generation for non-work-related things, then spend time refactoring it and cleaning up the mess.
AI is pretty decent but lacks the ability to write "maintainable" software, which is easy to extend, and replace. It works as a fantastic starting point in that it fills in all the boilerplate code.
Kind of having the same feeling as well the more I use ChatGPT and similar systems/models.
I wonder if this will result in a general decline in code quality. Or if it's going to help people become better IT professionals overall.
I mean, in order to iterate and make things better after first try/implementation, also requires understanding what you typed and it can be improved.
Exciting times nonetheless