Hacker News Reader

Comment by ares623 | Hacker News Reader

What I’d really like to see is a company that completely does away with the code review step. Because if you think about it a code review is already being done by the invoker of the LLM. That way the velocity will actually be faster. It feels like at the moment most of the velocity is blocked by the outdated notion of code review.

It’s also why these tools feel great in green field projects, since these ones don’t typically have any code review in place (i.e. one dev going brrr)

View on HN

We tried this as a bit of an experiment - not quite greenfield but a fork and modification of an existing project.

I was a solo dev going brrrr with no code review, using Cursor to make pretty sweeping changes and implement full features. I think because it was a fork I was able to go really fast because I could point the coding agent at existing patterns and say, "do this but different".

Occasionally I would test changes in staging but mostly would just promote them straight to production. I went on like that for ~3 months and the product ended up getting ~75k (daily!) users in that time with *no* outages or downtime (!!!)

It worked really well! Until I hit my scaling limit and we added more engineers to the team. A new hire on their second week on the job shipped a change that was also written with AI and caused a 7 hour outage before we realized something was wrong. I even *thoroughly reviewed the code* because I knew it was generated and I made a comment questioning the lines that caused the outage, but after we talked it over we decided it was fine and was only one tiny bit of a much larger change.

So I guess, AI will ship bugs but so will peer-reviewed code.

Nice. Sounds like a success? Was the experiment made permanent? If not, why not?

I would say it's a success, yes - it was great for quickly scaling up a theoretical product with a small engineering staff (just me lol).

Now that the product has matured, we've had to slow down significantly since we have a ton of paying users who expect SLAs and less bugs. This means more engineers, which means more testing and review, which means more process around what changes we're making.

I realize this is likely being facetious, but just in case - code reviews are so much more than just 'check the syntax and style of the code'. They check the intention, check the actual functionality, find issues on the larger scale that LLMs literally can't.

Yes, PRs start piling up because devs can vibe code them faster than they can be competently reviewed. This is a problem with the vibe code process, not the code review process.

I was being half facetious, yes. But wouldn't the invoker of the LLM be already doing a review in that case? It just feels a bit redundant, to have engineer one do a code review of LLM's work, and then have engineer two do the same review.

The business cost with PRs isn't the first review but another developer, it's the number of iterations on a pull request due to defects and change requests. The way I am trying to promote the use of LLMs with more junior developers in my team (I am the CTO) is to use AI-assisted tools (we used Windsurf and recently switched to Github Copilot) for a first pass, e.g asking the agent for a review and catching potential defect before involving the human reviewer.

This doesn't mean the human reviewer will need to spend less time reviewing, but potentially this PR will be merged faster with on average a lower number of iterations and improved code quality.

I do have in my team some senior developers that are excellent, and it's very very rare I catch an issue in their PRs (maybe 1 out of 50). But I also have greener developers for who the ratio is way higher (like 8 or 9 out of 10) and this means repeated context switching for the reviewers.

I hope this is a joke...

Partially serious. I mean what’s the point of being able to open 100 PRs in one day if your coworkers can only reliably review 5 of them?

Not just that but PRs are about a lot more than syntactical correctness, it's also about design. Just this week we've had two PR catch bad design choices that came from a fundamental misunderstanding of the problem space. Even if the AI wrote perfect code the first time everytime it wouldn't matter in those cases.

Reply on news.ycombinator.com