Ask HN: What's your experience with AI-based code review tools? | Hacker News Reader

Ask HN: What's your experience with AI-based code review tools?

19 pointschw9e12 comments

I've seen a lot of AI code review tools hitting the market, and I'm curious to hear from people who have tried them out.

Do they speed up reviews? How?

Do you still need human reviewers, or can you directly land code that the AI says passes?

Do they lead to higher quality code? Do they catch important enough things that the review is worth it, or is it mostly minor things?

View on HN

A lot of value in code reviews comes from knowledge sharing and collaborating on code. It is close to a pair programming exercise that many people in the team can partake in asynchronously.

I’m sorry for not answering many of your questions, but I do think it’s not worth it, at least in a professional context.

For personal use, it might be different.

I'd treat it more as a linter. You deal with it before submitting review for a human.

Sure for knowledge sharing about tribal knowledge or undocumented plans I see the value of having a person reviewing, but a lot of other stuff that comes up in pair programming can probably also be managed by AI like what are best practices, a bug you might have missed, evaluating alternative solutions, etc.

People are incredibly varied on code reviews too, I can see where some standardization on what should be blocking and what is best left as a suggestion could accelerate things in some cases and prevent issues in other cases.

In my experience, automated AI review can definitely find and identify bad patterns or practices in code or popular SDKs, but our company's tech stack is highly distributed and there is a lot of business domain logic that the AI simply can't really predict (at least not with the current generation) how new code may interact with it on a macro level.

Which code review tools are people using?

I think it needs to know every conversation that happened in the company to provide meaningful feedback that is above generic coding suggestions. But then it makes onboarding new people and getting them familiar with the project much easier. Businesses will have a lot of incentives to do that.

Management wouldn't risk being called on their bullshit and contradictions.

It helps catch the most obvious bugs. Like the algorithmic bugs. If the comment string says something and the code does not do that, that's caught fine. But it is not so good at catching domain related bugs. I think that's to be expected because these AI-tools don't have the knowledge of the domain and the hidden assumptions that come from experience in a domain. It could improve in future if the AI-based tools are trained on a lot of design documents + code that solve domain problems.

Which begs the question why isn't it flagging a lack of comments on what the function/class does.

It helps catch the most obvious bugs. Like the algorithmic bugs.

IME, algorithmic bugs are rarely obvious.

It’s more fun to make your own ai linter then to buy a service.

python has an ast library that would be pretty cool for recursing through a code tree.

I also though it would be cool to add a database of class/functions, GPT comments, params, file path.

I thought it might be cool to have it ask questions about the code also. And store answers.

Reply on news.ycombinator.com