Hacker News Reader

Comment by torben-friis | Hacker News Reader

>A caveat: Lines of code is an imperfect measure, as it measures quantity over quality. So 8× lines of code/engineer/day in the second quarter of 2026 is almost certainly an overstatement of the true productivity gain. Nonetheless, it indicates an acceleration. At Anthropic, we don’t reward people for how many lines of code they write; rather, team members are producing more code simply because they’re using AI systems to write more code.

What about the hypothesis that AI is generating more verbose code? I just see the text pretending to acknowledge "LOC != Productivity" and then using it as a metric anyway.

View on HN

One of my co-workers just asked me to review his pull request that was all AI generated. 600 files were touched, over 40k lines of code added.

I'm sure he thought that was a crowning achievement, proof that AI can enable 10X developers, after all, what engineer could write 40k lines of code in a week?

I declined to review it, stating that I couldn't possibly vet 40k lines of code, and wouldn't put my reputation on the line to stamp the work as good. The PR nagged me for 2 weeks from my todo list and then disappeared. I don't know if he found another dev to get an approval from, or if the PR was abandoned. But I know for sure that him and I are on two totally separate islands around the value of LLMs.

I declined to review it, stating that I couldn't possibly vet 40k lines of code

Gee, that sounds like a job for Claude if there ever was one.

You're absolutely right!

And how would you verify that the review was accurate?

Same way that I would trust your review to be accurate. Because the reviewer has built a reputation for correctness.

Its not Claude doing the review. Its a human doing the review, but using Claude to do the reading. Its still on the human to ask the right questions to Claude.

For large changes that are not straightforward and include architectural decisions, I wouldn't trust Claude enough to not read most of the code myself. I'll have to read it to be able to understand it and ask about the decisions in detail anyway. And when I start to understand it, it's not uncommon to find out that the solution can be improved and simplified in many places, and after iterating, 25-30% of code disappears.

And trying to just hand-wave it to Claude, to somehow "improve it" or "simplify it", without detailed questions hasn't been very successful. It can work for some things, though.

So you'd produce the code using Claude, and then use Claude to verify it? Would you accept my review of my own code?

Depends. Do you take pills that let you forget that you wrote the code so you can review that same code with fresh eyes that haven't seen that code before? Though you could just use ChatGPT to review the code that Claude wrote if that's really the issue.

Claude to the rescue

Claude Van Damme ?

I'd prefer Chuck to the rescue but I guess it's a cultural preference.

You can use Claude for that

Ask it to prove it.

My approach for AI-first code review, or really any kind of AI technical opinion, is that if the claim AI made is both important and not obviously true at a glance, it has to prove it to me, and keep trying until I'm convinced or can spot an obvious mistake in the proof.

With reviews, this is usually the case where AI is making a claim that something in the PR will fail because of some assumptions or behaviors in code outside of the PR - e.g. "this change will fail in scenario X, because foo is null in this case, because the SQL query doesn't populate it when bar == quux, and it gets propagated as null through the JSON deserialization (optional field)...", where all the SQL and JSON parsing was not part of the code under review, and "bar == quux" is some weird domain special case.

Stuff like this is both critical, and there's no way for me to judge it without an expensive context switch. So I learn to ask for a more detailed walk-through once, and if that doesn't make me "see" it, I just ask it to reproduce it with tests, and confirm it's a real problem. Reviewing the reproduction is usually enough for me to either "see it" or accept they're probably right and ask the author to recheck it.

(Why not jump straight to "reproduce it" for every finding? Because it still takes time to have AI do the repro. It's cheaper than a deep context switch, but not free.)

At work we had copilot. It said "the diff is too big to review"

That's a process problem at your company - no developer should be proposing branches over 1k loc (or whatever your agreed tolerance threshold is) without a very good reason, vibe coded or not.

> no developer should be proposing branches over 1k loc

I've seen that reaction many times. It seems to work well enough when someone is maintaining existing code. However, greenfield projects can often require literally orders of magnitude more code to deliver something that can be integration tested.

The first step is to break it up into a stack of commits. Each one must compile and pass its unit tests, of course. Keeping it under 1k loc of released executable code is usually easy, but often becomes difficult to impossible if you want well commented code with excellent unit test coverage.

Assuming you have kept all your commits under 1k loc, there is still the problem of whether you present them in one PR, or as a stack of PRs. The issue with a stack is why an API is designed a certain way often isn't evident until you see how it's used. Responses to PR comments are explanations that point to later PRs in the stack, which is irritating for both the reviewer and the author.

I haven't found a good solution. I'm not sure there is one.

I mean, you're not creating the api from thin air as you write code right? Usually you'd have some larger doc about the project with the design you can point to.

It isn't about small or big, it's about cohesion of the changes.

I prefer a big feature to be one big PR rather than a lot of small ones.

We had a dev do a big feature with a ton of small PRs, each one was individually impossible to review because each concern was out of scope for the small PR and "would be fixed in later PRs". Once it all came together as as whole, the big picture was a total horror show and I had to rewrite basically the whole thing.

In order to review those small PRs properly, each time I would have to read and understand all the current code so far from the beginning. Without that, each small PR individually looks OK because you won't remember the other PRs from weeks back that already duplicated what the current small PR does for example.

>> I prefer a big feature to be one big PR rather than a lot of small ones.

Yes, same, and I genuinely do not understand the insistence that PRs should not be above a certain size. I think most people are under the (misguided and wrong) impression that a PR review should take less than the time it took to write the code, and therefore allocate no more than 15-30 minutes per review. So when they come across a large PR they find themselves at a loss.

Fully agree. If you want to follow the thought process through a large PR, review each commit (assuming, of course, the author made reasonable commits) on its own.

> no developer should be proposing branches over 1k loc

I completely agree with you. But I am afraid we are losing the battle.

I am seeing people repeatedly sending out gigantic PRs full of slop, code with mistakes that they would never have made if they were hand coding it. And they don't care. It's sometimes surprising if not horrifying to find that the colleagues you have worked with for years don't care about quality at all -- almost despising spending time reviewing their own code. Yet they have the audacity to send out code reviews.

Same here. A co-worker touched a few hundred files in a PR and asked us to review. They merged it directly to main when nobody approved it. (The repo was not set up to enforce PR approval.)

I don't personally use that feature, and I couldn't care less at this point. If our customers are frustrated by the bugs, at least my name is not on it.

The challenge is that you may not have customers, and thus a job, if that continues

His job will be to fix all the mess that other people created.

Crazy they merged into main holy moly

This is a branching point. One dev would find someone else and convince them to approve it. Another would redo the task (code is cheap now, right?) in a PR stack that can actually be reviewed, cleaned up etc.

I hope they were the latter.

You could surely check on the status of that PR.

A former coworker sent me an AI generated PR to review and I just said NAK after the first two issues I found and I said to not send me AI slop to review.

They went to HR who said I am more senior and I should act as a mentor (they had my same work title and were probably making 4x more due to being in USA) and I just no longer reviewed anything from them until I changed jobs.

People actually go to HR for such trivial things?

All the time. And HR are generally bored and enjoy some drama.

My review would have been along the lines of:

'Please split this PR into smaller ones'. I would even sketch which groups/phases would make sense, perhaps with the help of AI.

I just watched copilot today turn a 8 line fix into 500 lines, so, yeah, verbosity is a big side effect

It occurs to me this pattern might be the average code we humans have produced. We all have made those quick fixes, copy-pastas, and dirty hacks... they learned it somewhere! I also assume that some of the behavior is an artifact of their training regime.

So with LLM outputting average code, and people using LLM more and more, I guess the average code will become worse over time ?

Not advocating for AI code slop--but if AI coded software works correctly, maybe it doesn't matter? Except sometimes when a specialist will have to get involved. Not a perfect analogy, but most people don't write assembly these days--they have a compiler do that. Assembly still has a place, but it's a specialist task.

> if AI coded software works correctly, maybe it doesn't matter?

The problem isn't the amount of code, it's how fitting/unfitting the abstractions are. Wrong abstractions are bugs in waiting. If there's much code with wrong abstractions, future change becomes difficult.

Source: me, I've created many bad abstractions and they led to much pain...

Yeah. Its kind of strange - claude is great at some tasks, but it seems really rubbish at coming up with good abstractions a lot of the time. I've often caught it making a conceptual mistake (like "X cannot do Y") - then spending hundreds of lines working around an issue that doesn't actually exist.

Its also really bad at inventing and leaning on invariants. I make rules in my code all the time - "by the time we get to path X, we know Y and Z are true.". In aggregate, these invariants make code simpler and easier to reason about. But claude doesn't do that. It just kind of - slops through and adds bespoke "just in case" workarounds all over the place. Every time I read through code its written - without fail - I find bad design / architectural choices.

Maybe mythos will change this. But for now I've slowed way down on my claude code usage. You can't build a skyscraper on a foundation of mud.

Yeah, I have a contract project for a webapp/integration to legacy Excel tool with an API endpoint for exchanging data with Excel. Over time, I notice issues or need to add functionality in the data processing and hadn't been closely watching the code changes Claude Code made to the API as long as it worked as expected/tests passed.

When I eventually read through the current state of the upload processing code it was like an absurd tree of checks on checks on fallbacks on triple checks added in response to whatever bug I reported in a bizarrely additive way and could be massively simplified (which would also make it less brittle to edge cases that then demanded more checks and workarounds).

The other issue is that for the upload API, there is documentation but not for every little bug or edge case so each time the model "wakes up" and loads everything into context it sees that crazy web of checks and edge cases as the only source of truth for the API so is hesitant to touch anything unless 100% necessary which then leads to more conservative behavior of additive code which makes the problem worse over time.

Codex seems a bit better but I still have to guide it towards proper abstractions/refactors to avoid that piling on cruft effect.

They have no illative sense like we do.

But the truth is: it doesn't work correctly. I see quality of software dropped significantly.

At work we are integrating with third party platform to automate excel-powered calculations. It is awful. Rendering the table in browser takes 10s or one click on Export button will throw backend in OutOfMemory state.

Ai mirrors the code around it. So if there is bad code or good abstractions, it's going to do the same. Even with good code, it will do bad things, you have to remain in the loop and catch these. It can write good code, it just needs nudging.

I don't disagree there is a lot of slop being produced right now, but I'm still optimistic in the long-run.

More verbose code takes up more space in the context. It's harder for humans to review, but also harder for future AIs to edit. Unless you manage to keep the AI to firm module boundaries & have it replace modules wholesale it's not really equivalent to how assembly gets replaced wholesale when a compilation unit changes. Compilers aren't editing the `.o` files when you rebuild, they throw the old ones out & replace them. But when you prompt an AI it is reading & editing the source files, so excess verbosity in the source files is detrimental.

Well, if tokens = cost, and verbosity = more tokens, then smaller code is a financial (and human!) win. Although I'm worried vibe coders are just going to have LLMs modify minified code in caveman mode so they can have 100 agents in a swarm..

On a more serious note, I wonder if this might eventually encourage people to use languages that are a little harder to write but much more concise (functional languages for instance). When you're paying per-token enterprise bean java style verbosity totally sucks

There is a belief that everyone is just taking whatever the LLM (really agents now) outputs. This is not the case anywhere I work. We use human oversight to have it iteratively improve the code. The average quality is going up.

Do you have some metrics to back this up ? Because from what I can see with my own eyes, outages everywhere, security holes everywhere too, doesn't seem that things are improving..

There are insufficient metrics for both directions, and while you can find evidence to support either position, this discounts two things

1. Things are evolving. The models, and especially the harnesses, are getting better. There was an inflection point at the end of last year, so anything before that is no longer relevant to the discussion. Probably anything before now, since we've had about 6 months with real agentic engineering and things are starting to become clearer.

2. Application and effectiveness is not equally distributed. This is the newest and most significant technology humans have created. We are still building and figuring it out. Some people are better at it, some people use it rather unwisely.

In my case, where I see it most often is when the LLM has to rework something multiple times, and the feedback loop is vague (especially when all I have to give it is "no error messages, but it's still broken"). It seems like after the third or fourth try it just kinda goes off the rails. I find that the one-shot quality tends to be a little better, if the slot machine happened to work correctly that time.

You shouldn't be using an LLM directly (web chat style). A proper harness allows an agent to see the errors itself and correct as needed. You can the correct it at higher, more meaningful levels.

My experience is in doing this with Claude/Codex/OpenCode with a pretty rigorous setup (AGENTS.md/CLAUDE.md for specific subfolder rules, strict compile/test/lint rules. This isn't me copy-pasting from web chat.

I've been building a general linter tool to help keep repos in a consistent and clean shape when working with AI [0]. You can define repo and file layout, structure, hygiene rules and have them checked in pre-commit, in ci, or manually. It also integrates and plays well with AGENTS.md - allowing exporting agent instructions from the alint rule config [1].

[0] https://github.com/asamarts/alint

[1] https://alint.org/agent-friendly-linter/

If you can make it 800 you can claim to be a 100x engineer!

Real question is, are you a 100x prompter?

Missed opportunity! I obviously have skill issues.

So the more rigorous studies about AI-assisted coding productivity addressed this by keeping in place all other software development processes, including the same code review and quality standards, and only measuring throughput (PRs, LoC) before and after AI was allowed.

Hence the intepretation of this 8x number depends on whether (or how much) Anthropic engineers have changed their quality standards and development processes. They don't tell us, and I am not aware of any other indications we could use to make a judgment.

However, we can still do some theorycrafting! I'm convinced that to fully realize the potential of AI-assisted coding we need to revamp all the dev processes, especially how we validate code, and it would be foolish of Anthropic not to do so (unless they were conducting a rigorous study, which they don't claim to have done.)

My hypothesis on the future of software validation is nothing fancy, we simply want much, much more automation for tests, observability and other bespoke verification methods than we traditionally had. But then validation code will also contribute to the LoC! My observation so far of personal as well as some "vibe-coded" open-source projects is O(LoC production code) ~= O(LoC test code). So as a SWAG the upper bound could be something like a 3 - 4x speedup, which is still remarkable.

All bets are off if code quality standards are not the same.

Exactly. If AI is going to start being graded on how many LoC it generates- oh, I'm sorry, how much it "accelerates", than guess what newer models will start doing more of?

Surely they can train AI on the signal to change as few lines as possible. Indeed, this is something I'd want to have control over when making requests. In a traditional UI, I'd imagine some kind of slider between "fewest lines" and "be bold".

I've been having some success asking Claude to run sloccount after each change. Seems to help a little, though it's prone to forgetting over a long session.

I'm actually hopeful that the recursive code training will improve quality over time. I'm definitely producing higher quality code, tests, and docs. It does take attention and oversight, iteration and refinement, one cannot just let these things loose on a code base and expect good things to happen. You have to leverage them to make the good things happen.

I don’t understand how lines of code matter at all for scary LLM core capabilities. Does the transformer architecture get better with more lines of code?

My impression was that LLM training codebases were 99% resource management and only a few lines actually implement the core training algorithm, which is where 100% of the intelligence comes from. Data, not lines of code, are the constraint.

After training you can adapt the intelligence in various ways, and that takes a bunch of lines of coded too. But you cant raise the intelligence ceiling again without another training run. So where is the scary recursive part?

Yeah, they assume that "productivity = k * LOC" where k > 1

very flawed

Could just be more tests? :) Which is good for code quality in general and reduces support burden, but doesn’t lead directly to more features

Just imagine the productivity gains from using LLMs to rewrite Kotlin codebases in Java!

AI generates code that mimics the existing code. If your code is terse and comment-free, then the agent’s code is too. The times I’ve seen Claude drift into a default “house style” it generated like 1 comment for every 10 LOC or so. It’s a far cry from the GPT-3 days that littered every line with the journals of Captain Obvious.

That is definitely not my experience using Claude Code with Opus. I work in a very sparsely commented code base, and the agent produces substantially more comments than the surrounding code.

Reply on news.ycombinator.com