1. First thing I noticed was the lack of autocompletion when entering @<filename> (or #<filename> with Github Copilot). Makes it really difficult to talk about files, since I don't always remember the full name. Had to go to VSCode to find it.
2. I assume, even though I selected Claude, it doesn't read CLAUDE.md? Does it have an equivalent? It had trouble navigating my repository. Pointing it to CLAUDE.md in the prompt fixed it, though.
3. For the second task, it created a plan (without me asking for one) and let me approve it. But then it marked the whole plan completed in the first few seconds, except the last item, before it wrote a single line of code. Or execute anything (the plan included what to edit, and what to test/run).
4. The generated code was fine for both tasks. The first one had about 20 changed lines, and the second ~100 lines + 300 lines of tests. It was very thorough in testing its changes, like the latest versions of Claude Code.
5. The bad part: implementing the plan for the 100 LOC change took about 13 minutes (unfortunately it doesn't show the exact duration when finished; it was at >100 tasks and over 770s). To compare, I stashed the changes, tried it on Claude Code with the same prompt, default model (Opus 5), auto mode. It did not create a plan and finished the task in 12 minutes. Claude's code was a bit simpler, and I prefer it over Bullet's by a small margin.