Hacker News Reader

Comment by jdietrich | Hacker News Reader

>I often run into LLMs writing "beginner code" that uses the most fundamental findings in really impractical ways. Trained on too many tutorials I assume.

In the absence of any other context, that's probably a sensible default behaviour. If someone is just asking "write me some code that does x", they're highly likely to be a beginner and they aren't going to be able to understand or reason about a more sophisticated approach. IME LLMs will very readily move away from that default if you provide even the smallest amount of context; in the case of this article, even by doing literally the dumbest thing that could plausibly work.

I don't mean to cast aspersions, but a lot of criticisms of LLMs are really criticising them for not being psychic. LLMs can only respond to the prompt they're given. If you want highly optimised code but didn't ask for it, how is the LLM supposed to know that's what you wanted?

View on HN

In my experience the trouble with LLMs at the professional level is that they're almost as much work to prompt to get the right output as it would be to simply write the code. You have to provide context, ask nicely, come up with and remind it about edge cases, suggest which libraries to use, proofread the output, and correct it when it inevitably screws up anyway.

I use Copilot for autocomplete regularly, and that's still the peak LLM UX for me. I prompt it by just writing code, it automatically pulls into context the file I'm working on and imported files, it doesn't insist on writing an essay explaining itself, and it doesn't get overly ambitious. And in addition to being so much easier to work with, I find it still produces better code than anything I get out of the chat models.

> In my experience the trouble with LLMs at the professional level is that they're almost as work to prompt to get the right output as it would be to simply write the code.

Yeah. It's often said that reading (and understanding) code is often harder than writing new code, but with LLMs you always have to read code written by someone else (something else).

There is also the adage that you should never write the most clever code you can, because understanding it later might prove too hard. So it's probably for the best that LLM code often isn't too clever, or else novices unable to write the solution from scratch will also be unable to understand it and assess whether it actually works.

Another adage is "code should be written for people to read, and only incidentally for machines to execute". This goes directly against code being written by machines.

I still use ChatGPT for small self-contained functions (e.g. intersection of line and triangle) but mark the inside of the function clearly as chat gpt made and what the prompt was.

After 6 months of co-pilot autocomplete in my text editor feeling like an uninformed back seat driver with access to the wheel, I turned it off yesterday.

It’s night and day to what I get from Claude sonnet 3.5 in their UI, and even then only on mainstream languages.

I like copilot for completing one liners.

Also Copilot Chat in VSCode works like Cursor IDE if you provide #codebase in the prompt.

It depends on what you’re doing. I’ve been using Claude to help me write a web admin interface to some backend code I wrote. I haven’t used react since it first came out (and I got a patch randomly in!)… it completely wrote a working react app. Yes it sometimes did the wrong thing, but I just kept correcting it. I was able in a few hours to do something that would have taken me weeks to learn and figure out. I probably missed out on learning react once again, but the time saved on a side project was immense! And it came up with some pretty ok UI I also didn’t have to design!

Reply on news.ycombinator.com