There have been many exaggerated claims surfacing across Twitter and the blogosphere. What I found to resonate best with my own experience is that Copilot is okay on simple tasks, but actually misguides, confuses, and in fact breaks your flow of thought on anything beyond that. I like that sort of smart autocompletion and snippet retrieval for boilerplate, templates code, but for the business logic proper it’s just awful. It either doesn’t get it at all but wants to insert some crap and the IDE isn’t always helpful to recognize that it’s distracting you, or it hallucinates something that seems to fit at first sight but once you accept the suggestion moments later you have to undo it because you realize how far off that is, how it misrepresents the domain entities or logic, as if a junior dev suddenly injected that code into your shared file buffer in a pair programming session, while you were focusing. So while Copilot autocomplete is on, you always have to analyze what a gung-ho quasi junior dev knee-jerks into your file buffer.

As many have witnesses, the Copilot Chat has been just terrible. I give up on my hopes for the current wave of the AI evolution.

What is uniform across all LLM is the lack of nuance. Even when they manage to generate some domain specific characters that do make sense, there’s always lack of detail and nuance to the domain, regardless how you try to trick your prompts to retrieve something from a very specific context. I’m impressed by how much easier it’s become for me to get a digest of longer texts, but at the same time I’m disappointed by the quality of the results. It is very rare that I get what I actually ask for. It’s like talking to a mid level consultant who pretends to know everything but the output is rather questionable, and you just give up and seek to end the meeting.