Hacker News Reader

Comment by mixologic | Hacker News Reader

Worth noting that mcollina is a member of the Node.js Technical Steering Committee

yes this.

if there's anyone i would trust in exploring these avenues, it's him and the maintainers doing god's work in the nodejs repo in these past few years.

We call it a slip slop at work, it's ok to slip some slop if it's "our" slop :-)

> I pointed the AI at the tedious parts, the stuff that makes a 14k-line PR possible but no human wants to hand-write: implementing every fs method variant (sync, callback, promises), wiring up test coverage, and generating docs.

Is it slop if it is carefully calculated? I tire of hearing people use slop to mean anything AI, even when it is carefully reviewed.

Was 14k lines carefully reviewed? Seems unlikely.

Considering the many hundreds of technical comments over at the PR (https://github.com/nodejs/node/pull/61478), the 8 reviewers thanked by name in the article, and the stellar reputations of those involved, seems likely.

My mistake 19k lines. At 2 mins per line that’s (19000*2)/60/7=90 7-hour days to review it all, are you sure it was all read? I mean they couldn’t be bothered to write it, so what are the chances they read it all?

For someone’s website or one business maybe the risk is worth it, for a widely used software project that many others build on it is horrifying to see that much plausible code generated by an LLM.

When you review code, do you spend 2 minutes per line? That seems like a huge exaggeration of effort required

I probably review about 1k LoC worth of PRs / day from my coworkers. It certainly doesn't take me 33 hours (!!) to do so, so I must be one of those rockstar 10x superhero ninja engineers I keep hearing about.

Are your coworkers producing the code using LLMs? And what level of trust do you place in them?

For half my coworkers, their LLM code is better than their code.

That’s depressing. For 80% of my coworkers their LLM code is horrible. Only the seniors seem to use it well and not just spit out garbage

I think that goes back to whether they are programmers vs engineers.

Engineers will focus on professionalism of the end product, even if they used AI to generate most of the product.

And I'm not going by "title", but by mindset. Most of my fellow engineers are not - they are just programmers - as in, they don't care about the non-coding part of the job at all.

Depends - if it is from a human I find I can trust it a lot more. If it is large blobs from LLMs I find it takes more effort. But it was just a guess at an average to give an estimate of the effort required. I’d hope they spent more than 2 mins on some more complex bits.

Are you genuinely confident in a framework project that lands 19kloc generated PRs in one go? I’d worry about hidden security footguns if nothing else and a lot of people use this for their apps. Thankfully I don't use it, but if I did I'd find this really troubling.

It also has security implications - if this is normalised in node.js it would be very easy to slip in deniable exploits into large prs. It is IMO almost impossible to properly review a PR that big for security and correctness.

> I mean they couldn’t be bothered to write it, so what are the chances they read it all?

What kind of logic is this?

It’s much harder to read code carefully than to write it. Particularly code generated by LLMs which is mostly correct but then sometimes awful.

usually yes, but that's why there are tests, and there's a long road before people start depending on this code (if ever). people will try it, test it, report bugs, etc.

and it's not like super carefully written code is magically perfect. we know that djb can release things that are close to that, but almost nobody is like him at all!

The PR has been open for 3 months, and all the reviewers involved have actually read the whole code and are experts on the matter.

I carefully review far more than 14k LoC a week… I’m sure many here do. Certainly the language you write in will greatly bloat those numbers though, and Node in particular can be fairly boilerplate heavy.

Reply on news.ycombinator.com