Hacker News Reader

Comment by zozbot234 | Hacker News Reader

Sure but the OP suggests that these were minor gains, and that this limited scope for gains was necessary in order to preserve the quality standard that's long been expected in that FLOSS community. We aren't talking about either a 10x productivity gain or one-shotting entire new features from scratch.

This is arguably a key quote: "Then, it was time to read all the code, line by line. ... I found many small inefficiencies or design errors ... so I started a process of manual and AI-assisted rewrite of many modules." We should not underestimate that step: reading code line by line might easily require more time than writing it from scratch.

View on HN

Right, and those of us who advocate for a sensible approach to agentic engineering don't talk about 10x productivity gains or one-shotting entire new (production-ready) features from scratch either.

I remain unconvinced by the "faster to write it by hand than read it" arguments though. My experience throughout my career is that most people, myself included, top out at a couple of hundred lines of tested, production-ready code per day. I can productively review a couple of thousand.

"top out at a couple of hundred lines of tested, production-ready code per day" + " productively review a couple of thousand." + LLM agents that write code for you = apparent contradiction with your first paragraph.

Right, I don't think you can "productively review a couple thousand" lines of code per day. That would imply that the review step for this very patch only took a couple days in total (since the core code is described as 5k lines) which is rather implausible to say the least.

Both Simon Willison and Antirez said that using LLMs helped them, so it's kind of perverse to read them and conclude the opposite.

In particular, doing direct comparisons between metrics like that doesn't work. "Lines of code" isn't a good way to measure complexity of the code, and the amount of time it takes to review the code will vary quite a bit based on the use case.

There's a lot of diversity in what kind of code people write and just because it worked for someone else doesn't mean it will work for the kinds of problems you solve. It's anecdotal evidence that someone else found it useful, your mileage may vary.

The relevant question is whether it helped them 10x or anywhere close to what AI is now being sold as (supposedly even replacing software developers' jobs altogether and one-shotting complete products from a single prompt), or it's just acting as a kind of glorified autocomplete. So far we're clearly seeing the latter based on what both Simon Willison and Antirez are referencing.

Simon often says that its LLMs help him "write productive code", but most of the code he shows are python libs doing menial tasks. That's fine for tooling, etc, which is sometimes useful.

It would absolutely NOT work for production-code with critical concurrency / embedded / real-time stuff

Antirez wrote Redis. That is "production-code with critical concurrency"

To quote another of his posts:

> I fixed transient failures in the Redis test. This is very annoying work, timing related issues, TCP deadlock conditions, and so forth. Claude Code iterated for all the time needed to reproduce it, inspected the state of the processes to understand what was happening, and fixed the bugs.

...

> In the past weeks I operated changes to Redis Streams internals. I had a design document for the work I did. I tried to give it to Claude Code and it reproduced my work in, like, 20 minutes or less (mostly because I'm slow at checking and authorizing to run the commands needed).

From "Don't fall into the anti-AI hype" https://antirez.com/news/158

His summarized assessment from that very post: "...state of the art LLMs are able to complete large subtasks or medium size projects alone, almost unassisted, given a good set of hints about what the end result should be. The degree of success you'll get is related to the kind of programming you do (the more isolated, and the more textually representable, the better: system programming is particularly apt), and to your ability to create a mental representation of the problem to communicate to the LLM."

He's saying you should be writing up complex, highly detailed specs for the LLM to turn into code, stressing that it's critical to work in a self-contained and "textually representable" problem domain. This is not one-shotting complete products from a vague prompt. You're still going to need software architects, and they'll still be doing much the same work. Turning fully-specified design into code has never been a "10x" task, it was always regarded as a relatively straightforward, if often tricky part of the job. And the way he worked with Redis makes it clear that you can't take what the AI delivers at face value, either: you'll have to go through it yourself, and that will take time and effort.

First he didn't write Redis with LLMs, it was way before. Second I'm not speaking of him in that comment.

Also his whole blog is about how, in order to do a task, he would need to spec it properly, then do "code inpainting" with the LLM, then fix all the issues that he could spot only because he's a senior, then repeat, etc

Did you read it?

BTW the last day. I played with Claude to fix the simple things all by himself. Sadly we are on gitlab so I needed to tell him to use glab cli and I needed a little bit more time to setup than GitHub (why do they not support gitlab or other code forges…) However it is definitely a time saver in these 1-3 line changes. My workflow basically was:

Let the LLM cook by doing the issues one by one. In the meantime I could start reviewing them. Checkout, running, reading. It was definitely faster since it also correctly linked everything, etc. of course once the change goes beyond that it probably is not working. However I really thought that a good idea would be to check for that work and implement it according to the issue description and change a Mr once the description changes, at least as long as the Mr is 1-3 lines. And even if it does not work, I can just discard it.

(A lot of these problems are often typos that do not even need a checkout, they come in through bigger Mrs that should not be blocked because of them)

> Sure but the OP suggests that these were minor gains

When antirez says 'I ventured to a level of complexity that I would have otherwise skipped,' I don't think you can call that a minor gain. The alternative is likely something 'good enough' that leaves the community dissatisfied for months, and then after initial design mistakes become load-bearing the ideal implementation can never be realized.

He writes that right after saying "For high quality system programming tasks you have to still be fully involved". He's just saying that AI was useful to him for tedious special-case tasks (citing the addition of 32-bit support and fishing out bugs in new low-level implementations), that this required starting from a "huge specification" (not a one-shotted prompt!) and that he still had to go over everything with a fine-toothed comb afterwards. That's the farthest thing from the 10x silver bullet AI is now being sold as.

Exactly. LLMs are good at "code inpainting": you give them the structures / constraints / specs, and they write the boilerplate.

Then you need a senior to go realize the 100 mistakes it did, fix them, and iterate, which is why you can't replace "natural intelligence"

And there are real mathematical reasons why computers won't be able to break through "mathematical reasoning" on their own (indecidability, etc)

> We should not underestimate that step: reading code line by line might easily require more time than writing it from scratch.

Might?

Reading code you did not write is always going to take more time if you do it properly.

The author said "You know what was the biggest realization of all that?"

> For high quality system programming tasks you have to still be fully involved, but I ventured to a level of complexity that I would have otherwise skipped. AI provided the safety net for two things: certain massive tasks that are very tiring (like the 32 bit support that was added and tested later), and at the same time the virtual work force required to make sure there are no obvious bugs in complicated algorithms.

Reply on news.ycombinator.com