Comment by joshstrange | Hacker News Reader

Comment by joshstrange | Hacker News Reader

Parent Full thread

> LLMs bridge layers illegibly and probabilistically.

I’m not sure I agree but I want to make sure we aren’t talking past each other.

Using an LLM to write code does not have this problem IMHO.

Using an LLM as part of the code does have this problem.

I’m not writing off the usefulness of calling out to an LLM from code but it does bring “illegibly and probabilistically” into the mix whereas using an LLM to write code that’s human-reviewed/modified does not.

In fact, you should be unable to tell what I wrote vs what the LLM wrote if I’m doing right (IMHO). “Clever” code is always bad no matter who/what wrote it.

The only time I’ve been asked “did you use <insert LLM> to write this?” Was due to the speed at which I completed something or because it’s in a language that the person asking the question knew I didn’t know very well.

LLM's can be good at guessing the code you want, but I've seen them guess the comment or documentation correctly, then the implementation is the opposite of what you asked. It also generated tests which were for the incorrect case, with comments showing the opposite of what you wanted.

For somebody new at the language or new to programming in general, that difference can be imperceptible until it gets to a review.

I've also managed to spot people using an LLM to write stuff due to them doing things which won't pass linters or code tests.

The subtle differences between comment/code can absolutely be a problem, I’ve fallen prey to it as well but as long as you test/execute the code you’ll catch it if you didn’t catch it from reading over the code. I often reject anything overly clever or that I can’t reason out quickly when it comes to LLM generated code.

As for not passing linters or code tests that’s unacceptable behavior from an employee. If that happened more than a few times I’d be having a talk with the employee followed by showing them the door if they continued to do it. I’d behave the exact same way if no LLM was involved.

I do wish LLMs like Copilot could use the lint rules and/or tests to inform the code they generate but thankfully it’s normally a keystroke away for me to reform a block of code to match the style I prefer so it not the end of the world.

To use LLMs (or any tool) you have to be able to show you can use it correctly, safely, effectively, etc. If you can’t do that then you have no business being in a coding role IMHO.

Reply on news.ycombinator.com