Is the code audit included in that 2 hours?
But, it did require passing tests
Most of the changes in the end were relatively straightforward, but I hadn’t read the code in over a year.
The code also implemented some features I don’t use super regularly, so it would’ve taken me a long time to load everything up in my head, to fully understand it enough, to confidently make the necessary changes
Without ai, it would have also required a lot of google searches finding documentation and instructions for setting up some related services that needed to be configured
And, it would have also taken a lot more communication with the people depending on these changes + having someone doing the work manually while the scrapers were down
So even though it might have been a reduction of 15hrs down to 1.5hrs for me, it saved many people a lot of time and stress
But, from my experience now, I’d happily use AI to build the tests
At the end of the day: 1) a human is the ultimate evaluator of the code results anyway, 2) the thing either works or it doesn’t
Personally, I do try to keep a comment at the top of every major file, with a comment with bullets points, explaining the main functionality implemented and the why
That way, when I pass the code to a model, it can better “understand” what the code is meant to do and can provide better answers
(A lot of times, when a chat session gets too long and seems like the model is getting stuck without good solutions, I ask it to create the comment, and then I start a new chat, passing the code that includes the comment, so it has better initial context for the task)
If the training data contains some mistakes often it will reproduce them more likely.
Unless there are preprogrammed rules to prevent them.
As a side note, most good coding models now are also reasoning models, and spend a few seconds “thinking” before giving a reply
That’s by no means infalible, but they’ve come a long way even just in the last 12 months