Ask HN: How much efficiency improvement would you estimate Copilot gives you? | Hacker News Reader

Ask HN: How much efficiency improvement would you estimate Copilot gives you?

I work at a tech company with about 400 employees.

The company rolled out access to GitHub Copilot for everyone about a year ago.

After a year they did a survey of employees who used it and asked to estimate what percent efficiency gain they get from using it.

What surprised me is the average across responses was around 10% gain in work efficiency.

With all the talk of AI replacing programmers I was surprised to see it that low.

Does that match your experience?

View on HN

I've tried to calculate this.

I used copilot for 14 days and every time I used copilot and found it benefits me (saved me time), I wrote a check mark on a sheet of paper. Every time I tried to use it and it led me down a rabbit hole or simply didn't do much, I wrote a minus sign.

In the end there were roughly the same number of checks and minuses. Granted-- it wasn't the best experiment in the world and quite subjective.

I don't use copilot but Claude integrated in Spacemacs through gptel.

On difficult things (new conceprs I don't already know): -100% efficiency. I just end up reading the manual/code and figure out things by myself.

On day to day things: +20% efficiency, mainly because it's a faster search/reminder.

On simple things (like boilerplate): +300% efficiency. It eliminates the tedious, boring bits for me so I have more time and brainpower to focus on the first category.

How I reached these numbers? gut feeling.

How I get consistent results? I add project files as context and then write as detailed prompt as I possibly and reasonably can (this does take time). Rubbish in => rubbish out, so it's important to prompt well. I've found out that I need multiple successive prompts to get good results with refinments, rewriting in place always gave me poor results.

Overall, it's a boost but since nowadays I tend to spend my time in the first category mainly, I don't end up getting that much out of it. The 10% people claim seems right to me.

I guess I'm more motivated to do the boring bits as I'm not the one actually doing them, just reviewing, and they fly by faster, but how do you even quantify that? enjoyment points?

I think that AI won't replace programmers, at least for now. All those codes written by AI still need review and more attention within security. Here is a good article about Copilot: https://gitprotect.io/blog/github-copilot-introduction-an-ai...

10%-ish is about right. In Fred Brook's No Silver Bullet essay he makes the point that software development has multiple phases, say:

* requirements gathering

* design

* coding

* testing

* deployment

and suppose these all make an equal contribution. If you managed to make a tool that can do the coding in 0% of the time you still have to do 80% of the work!

Reply on news.ycombinator.com