I do think it's a pretty cool demo, but I have to say I didn't love that the PR claims Sweep did "manual testing" of the fix. Additionally, sweep reviews the PR and claims that the function is correctly implemented. A sibling points out that there's actually no testing added or done, and there's also issues with the implementation itself. This appears to be a general issue with GPT4 based products, they are extremely self-confident in their language. Presumably this stems from the overall training to work well as a chatbot.

It's very cool that it inferred the right place to make the change and the steps of finding relevant code, making a plan, then doing it are things I wish all my junior developers would do! This is certainly moving in the right direction.