Hacker News Reader

Comment by simonw | Hacker News Reader

If an LLM hallucinates a method name you'll find out the moment you try and run the code.

Code has a built-in form of easy fact checking, which makes it one of the most appropriate applications for LLMs. It's much harder to spot a hallucinated fact in a paragraph of prose than it is to spot a hallucinated API method.

The skills you most need to develop in order to take advantage of LLM assisted programming are code reading, code review, manual and automated testing and being really good at thinking of edge-cases that might not be covered.

It turns out these are important skills for being a great developer already - LLMs just force the issue on them a little more.

View on HN

Due to the nature of my current work I haven't really used GPT for coding yet, but I keep wondering isn't it easier to write code than to read and truly understand it? So how much development time was really saved, if I still care about off-by-one errors or correct identity checks in hash maps or all those edge cases I probably should care about? Those are all things much harder to spot reading than writing the code.

So I keep wondering if we just save time by introducing more unknown bugs using GPT?

I guess this also has a lot to do with what code is written. I would be much more concerned with a system level C++ library than some JavaScript CRUD.

Your comment is a great example of how intelligent humans hallucinate!

Gpt-4 often handles errors well. The generated code is easy to review if you understand what you asked for (if it generates tests and examples too-which it can). Etc

People have different experiences with any given thing. You might find your TV remote intuitive and easy to us. Your grandparents might find it overwhelmingly difficult. Which is true?

Yeah, sorry for the snark. I was mostly referring to how the comment I replied to said they hadn’t actually used GPT for coding, but had lots of ideas about its limitations etc.

If you read closely I am not talking about the limitations of GPT, but about the limitations of the developer having to fix code of others (GPT essentially being another developer). I guess it depends on the complexity of the problem. A lot of stuff is very easy to review and for me other stuff needs more time to understand than it did to write.

I have absolutely seen Copilot generate valid C# code with a nasty logic bug. You really need to carefully review every line, and at that point I would usually rather just write it myself.

I'm sure there are good use cases, but I just haven't seen anything life-changing in my experiments with various LLM products. The only thing I use day to day is IntelliCode, which saves a bit of typing here and there.

If the LLM hallucinates a Python package, someone else might notice that and sneak in a malicious version of that hallucinated package.

Or if it hallucinates a method name, it might be in a code path that goes untested. How often are people using these tools to also write comprehensive test suites?

Like I said, you have to put the work in if you're going to use these tools effectively. I write a LOT of tests both with and without the help of LLMs. I try hard never to leave any code path from an LLM (or from code I write myself) untested - always manually and usually through automated tests too.

If you don't do that, the chances LLM-generated code will introduce weird bugs are high. But the chances you yourself (or your coworkers) will introduce weird bugs are high as well - QA, code reviews and testing are important no matter what helped write the code.

> If an LLM hallucinates a method name you'll find out the moment you try and run the code.

What about when it confabulates a module name that's been squatted by a malware distributor?

One of the things you need to learn to do when working with LLMs is to avoid blindly installing packages it suggests.

Just like if someone opens a PR against your project on GitHub you should review the dependencies they are trying to bring in as well.

While it is true that code has built-in fact checking, i would expect AI to actually do the fact checking implicitly. They are all logic in the end.

I often see GPT/Gemini propose code solutions that refer to non-existing libraries or methods inside of these libraries. Perhaps the solution is to use specialized AI for coding is more advised.

And dont use the generic ones. However these hallucinations are still facts. The issue is that humans are also prone to hallucinations; this is perhaps the most challenging aspect to solve in AI. Everything the AI says must be vetted/fact-checked.

Back to coding with AI: i noticed AI, as described by the original author, is perhaps best integrated as an assistant. It works well when given enough context and a simple task that could otherwise take hours to complete but is now in seconds. So think of:

- given this function: …

- write test cases that covers the most common inputs.

LLM’s are still long way to go to replace programmers. But at the pace it’s going, it feels scary sometimes.

Reply on news.ycombinator.com