Did GitHub Copilot increase my productivity?
trace.yshui.dev
trace.yshui.dev
I had a ton of headaches from blindly accepting it's output when I'm in a rush and then later on realizing that it made my code dumber and reduced functionality. There were nuances as to why I was coding things the way that I was, why I used solution C, and not solution A, when A was the most obvious and rote, but broke features.
To be blunt, it's perfectly reasonable. They didn't even get that proclaimed "one percent!"
How does one even meaningfully measure this? No matter.
In their case, instead of saving time or effort, it was adding. It was a removable cost.
If one really wants to dwell, there's game theory. Could it be made to be effective? Probably. Is it worth it? Don't know.
They told us 'no', we should listen for their situation.
At risk of splitting hairs, I need more than one percent. This isn't a vacuum, I'm not a spherical cow, etc.
Point being: wanting both my money and attention has a high bar of admission. Learning a thing has a cost, becoming dependent, and so on.
I'm well into rambling now, feel free to tune out. I'm not sacrificing autonomy for a pittance. I was fine before I knew about $OFFERING. Truth in advertising is idealistic, at best.
However in the case of AI, I am not sure it helps me code any faster. I have to do tons of code cleanup when it messes something up and I "forgot" the logic of how the application was working because I let AI take the reins.
Junior programmers will likely find LLM coding tools magical because they don't have the skills and experience to outperform the "AI." I liken that to my parents thinking I have a special genius with computers when I get their wi-fi to work again or show them how to search for an email message -- let's call it a skills issue.
More senior programmers may find LLMs handy for filling in boilerplate or dropping in a well-known implementation of an algorithm or code snippet. Previously we would have to break stride and search StackOverflow and other places, or a book, or figure it out ourselves, so maybe we gain some time getting a chunk of code regurgitated for us, and lose some of that time carefully reviewing and testing what we got. A significant part of programming involves rote tasks, and in the most common domains (like web development) the code solutions tend to look the same. If GPT or Copilot can save me a few seconds of looking up how to reduce an array in Javascript, great, but I'm not sure I'll call that a big productivity boost because I don't measure my productivity in lines of code per minute.
To put it less charitably, the closer your skills approach 0x productivity (relative to the 1x - 10x programmers) the more an "AI assistant" will increase your output. But since we don't have a good way to measure productivity or compare it to past performance, much less to other programmers, we just talk about subjective experience made worse by Dunning-Kruger effects.
Opus seems better at code though so I am mixing them ; copilot for continues and Opus for big/new slabs.
From the article: unpredictable is correct but with a retry or two that is usually fixed. Sometimes of course it just cannot do it.
It’s slow; that’s true but a matter of time; if you see things like Groq and the others that are very fast, you see the near future will be faster than you can read, so you can create multiple or even self-fixes by the time you are even noticed it did anything.
Not OP, but this is the vast majority of the improvement I've seen.