Hacker News Reader

Comment by keeda | Hacker News Reader

That study gets mentioned all the time, somehow this one and many of the others it cites don't get much airtime: https://www.stlouisfed.org/on-the-economy/2024/sep/rapid-ado...

>This implies that each hour spent using genAI increases the worker’s productivity for that hour by 33%. This is similar in magnitude to the average productivity gain of 27% from several randomized experiments of genAI usage (Cui et al., 2024; Dell’Acqua et al., 2023; Noy and Zhang, 2023; Peng et al., 2023)

Our estimated aggregate productivity gain from genAI (1.1%) exceeds the 0.7% estimate by Acemoglu (2024) based on a similar framework.

To be clear, they are surmising that GenAI is already having a productivity gain.

View on HN

The article you gave is derived from a poll, not a study.

As for the quote, I can’t find it in the article. Can you point me to it? I did click on one of the studies and it indicated productivity gains specifically on writing tasks. Which reminded me of this recent BBC article about a copywriter making bank fixing expensive mistakes caused by AI: https://www.bbc.com/news/articles/cyvm1dyp9v2o

The quote is from the paper linked in the article: https://s3.amazonaws.com/real.stlouisfed.org/wp/2024/2024-02...

It's actually based on the results of three surveys conducted by two different parties. While surveys are subject to all kinds of biases and the gains are self-reported, their findings of 25% - 33% producitivity do match the gains shown by at least 3 other randomized studies, one of which was specifically about programming. Those studies are worth looking at as well.

It's worth noting that the METR paper that found decreased productivity also found that many of the developers thought the work was being sped up.

Yes, self-reporting has biases and estimating tasks is still a fool's errand, which is why I noted that the estimates from these surveys matched the findings from other RTC studies.

However, what doesn't get discussed enough about the METR study is that there was a spike in overall idle time as they waited for the AI to finish. I haven't run the numbers so I don't know how much of the increased completion time it accounts for, but if your cognitive load drops almost to 0, it will of course feel like your work is sped up, even though calendar time has increased. I wonder if that is the more important finding of that paper.

Reply on news.ycombinator.com