"Are there any bugs in the current diff"
It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.
"Are there any bugs in the current diff"
It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.
My experience is that it is about 10% harmful, 80% useless and 10% helpful. Which is actually great, the 10% is worth it, but it is far from a hands off experience.
By harmful I mean something like suggesting a wrong fix to code that works, it usually happens when I am doing something unusual or counter intuitive, for example having a function "decrease_x" that (correctly) adds 1 to x. It may hint for better documentation, but you have to be careful not to go on autopilot and just do what it says.
By useless I mean something like "you didn't check for null" even though the variable can't be null or is passed to a function that handles the "null" case gracefully. In general, it tends to be overly defensive and following the recommendations would lead to bloated code.
By helpful I mean finding a real bug. Most of them minor, but for some, I am glad I did that check.
LLMs complement traditional linters well, but they don't replace them.
That’s usually your signal that your code needs refactoring.
Every project worth making is unique. Otherwise, why not use something off the shelf?
For example, let's say you want to shuffle songs for a music player, you write your shuffling algorithm and it is "wrong", but there is a reason it is "wrong": it better matches the expectations of the user than a truly random shuffle. A LLM trained on thousands of truly random shuffles may try to "fix" your code, but it is actually the worst thing you can do. That "wrong" shuffle is the reason why you wrote that code in the first place, the "wrongness" is what adds value. But now, imagine that you realize that a true random shuffle is actually the way to go, then "fixing" your code is not what you should do either, instead, you should delete it and use the shuffle function your standard library offers.
The unusual/unique/surprising parts of your code is where the true value is, and if there is none of that in your codebase, maybe you are just reinventing the wheel. Now, if a LLM trips off these parts, maybe you need some documentation, as a way to tell both the LLM and a human reading that part that it is something you should pay attention to. I am not a fan of comments in general, but that's where they are useful: explaining why you wrote that weird code, something along the lines of "I know it is not the correct algorithm, but users prefer it that way".
I work with a high stakes app and breaking changes cause a ton of customer headaches. LLMs have been excellent at catching potential little bugs.