Hacker News Reader

Comment by anonzzzies | Hacker News Reader

Google and many people will tell you the same thing. So you must have a lot of lol moments in a day.

Gpt4 is quite good (better than most humans I ever met) at php, but bad at facts; don’t ask it facts, ask it to write code. That’s what you would ask a human (outside interviews).

Disclaimer; I am formally trained with proofs and proof assistants and I hate the current timeline where we ask ai to drivel up code, but I cannot say it’s bad at doing it; it just is not necessarily sound or even working code sure, but that’s the same as with most human first tries. Then you iterate and make better. My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.

View on HN

When saying LLMs are "good at writing code" there's a distinction between

"good at taking high-level English-language solutions to programming problems and translating them into low-level implementations via a specific language / design patterns / etc"

versus

"good at finding solutions to programming problems."

GPT-4 is indeed quite good at the former - and the former is what most enterprise programming work actually is. A lot of the hard part of professional software development is understanding the problem well enough to describe a solution in English: once you do that, writing the C# or Java is typically somewhat rote. Likewise LLMs are genuinely useful when you know exactly what you want to do with a 3rd-party library, but have to trawl through a bunch of API documentation to figure out the magic words.

All that said, LLMs still really suck at the latter problem: https://www.aisnakeoil.com/p/gpt-4-and-professional-benchmar... OpenAI's benchmarks fall off quite badly with actual programming benchmarks, compared to simpler tests of code generation. If it involves managing state, creating novel data structures, counting to numbers higher than 3, etc, LLMs just aren't smart enough.

You are right, hence my disclaimer. For paid work, it works fine and that is indeed enterprise kind of mindnumbing plumbing. For other work, I do not use it (but will try every few months if anything changed of course).

> So you must have a lot of lol moments in a day.

I do! I use code generation for non-work-related things, then spend time refactoring it and cleaning up the mess.

AI is pretty decent but lacks the ability to write "maintainable" software, which is easy to extend, and replace. It works as a fantastic starting point in that it fills in all the boilerplate code.

> My days of ‘I have proven it correct, now I just have to type it in’ are long gone, at least for things that pay for my bread.

Kind of having the same feeling as well the more I use ChatGPT and similar systems/models.

I wonder if this will result in a general decline in code quality. Or if it's going to help people become better IT professionals overall.

I mean, in order to iterate and make things better after first try/implementation, also requires understanding what you typed and it can be improved.

Exciting times nonetheless

Reply on news.ycombinator.com