Comment by ymyms | Hacker News Reader

Comment by ymyms | Hacker News Reader

"I find LLMs useful as a sort of digital clerk - searching the web for me, finding documentation, looking up algorithms. I even find them useful1 in a limited coding capacity; with a small context and clear guidelines."

I am curious why the author doesn't think this saves them time (i.e. makes them more productive).

I never had terribly high output as a programmer. I certainly think LLMs have helped increased the amount of code that I can write, net total, in a year. Not to superhuman levels or even super-me levels, just me++.

But, I think the total time spent producing code has gone down to a fraction and has allowed me more time to spend thinking about what my code is meant to solve.

I wonder about two things: 1. maybe added productivity isn't going to be found in total code produced, because there is a limit on how much useful code can be produced that is based on external factors 2. do some devs look at the output of an LLM and "get the ick" because they didn't write it and LLM-code is often more verbose and "ugly", even though it may work? (this is a total supposition and not an accusation in any way. i also understand that poorly thought out, overly verbose code comes with problems over time)

> "get the ick"

> even though it may work?

The first of those is about taste, and it's real, and engineers with bad taste write unstable buggy systems.

The second of those is about priority. If all you want is functional code, any old thing will do. That's what I do for one-off scripts. But if you plan to support the code at 2am when exposed to production requests on the internet, you need to understand it, which is about legibility and coherence.

I hope you do have taste, and I hope you value more than simple "it works" tests. But it might be worth looking there for why some struggle with LLM output.

For what it's worth, I use coding agents all the time, but almost never accept their output verbatim outside of boilerplate code.

See for some, "over time" means "the next guy's problem". That is true before LLMs of course. And that's even the prevailing school of thought in most organizations. And to some extent it is correct to accept some tech debt because otherwise you'll never get anything done.

For those who have been around a while, dealing with the "over time" of yesteryear is a daily occurrence. So naturally they are more averse to it. And LLMs seem to dramatically shorten the duration of "over time".

It seems you find LoC as a measure of productivity. This would answer your question as to why the author does not find it makes them more productive. If total output increases, but quality decreases (which in terms of code means more bugs) then has productivity increased or has it stayed the same?

To answer my own question, if you can pump out features faster but turn around and spend more time on bugs than you do previously then your productivity is likely net neutral.

There is a reason LoC as a measure of productivity has been shunned from the industry for many, many years.

I didn't mean to imply LoC as a measurement of productivity. What I really mean is more "amount of useful code produced to a level the human-using-the-llm determines to be useful".

To try and give an example, say that you want to make a module that transforms some data and you ask the LLM to do it. It generates a module with tons of single-layer if-else branches with a huge LoC. Maybe one human dev looks at it and says, "great this solves my problem and the LoC and verbosity isn't an issue even though it is ugly". Maybe the second looks at it and says, "there's definitely some abstraction I can find to make this easier to understand and build on top of."

Depending on the scenario and context, either of them could be correct.

LoC is a terrible metric for comparing productivity of different developers, even before you get to Goodhart's Law.

OTOH, for a given developer to implement a given feature in a given system, at the end of the day, some amount of code has to be written.

If a particular developer finds that AI lets him write code comparable to what he would have written, in lieu of the code he would have written, but faster than he can do it alone, then looking at lines written might actually be meaningful, just in that context.

I feel like it makes me more productive, but I am not even sure it does even with my light usage. How do we even measure it?

I also feel like it makes me more productive but measuring software engineering productivity is famously difficult. If there was an easy way to measure it, managers at bigco would have employed it with abandon years ago.

Reply on news.ycombinator.com