Hacker News Reader

Comment by pluc | Hacker News Reader

Every study I've read says nobody is seeing productivity gains from AI use. Here's an AI vendor saying the opposite. Funny.

View on HN

There's a few explanations for this, and it's not necessarily contradictory.

1. AI doesn't improve productivity and people just have cognitive biases. (logical, but I also don't think it's true from what I know...)

2. AI does improve productivity, but only if you find your own workflow and what tasks it's good for, and many companies try to shoehorn it into things which just don't work for it.

3. AI does improve productivity, but people aren't incentivised to improve their productivity because they don't see returns from it. Hence, they just use it to work less and have the same output.

4. The previous one but instead of working less, they work at a more leisurely pace.

5. AI doesn't improve producivity, people just feel it's more productive because it requires less cognitive effort to use than actually doing the task.

Any of these is plausible, yet they have massively different underlying explanations.... studies don't really show why that's the case. I personally think it's mostly 2. and 3., but it could really be any of these.

Why not all? I've seen them all play out. There's also the people that are downstream of AI slop that feel less productive because now they have to clean up the shit other people produced.

You're right, it kinda depends on the situation itself! And the downstream effects. Although, I'd argue that the one you're talking about isn't really caused by AI itself, that's squarely a "I can't say no to the slop because they'll take my head off" problem. In healthy places, you would just say "hell no I'm not merging slop", just as you have previously said "no I'm not merging shit copypasted from stackoverflow".

I think it's 5.

I was very impressed when I first started using AI tools. Felt like I could get so much more done.

A couple of embarrassing production incidents later, I no longer feel that way. I always tell myself that I will check the AI's output carefully, but then end up making mistakes that wouldn't have happened if I wrote the code myself.

This is what slows me down most. The initial implementation of a well defined task is almost always quite fast. But then it's a balance of either...

* Checking it closely myself, which sometimes takes just as long as it would have taken me to implement it in the first-place, with just about as much cognitive load since I now have to understand something I didn't write

* OR automating the checking by pouring on more AI, and that takes just as long or longer than it would have taken me to check it closely myself. Especially in cases where suddenly 1/3 of automated tests are failing and it either needs to find the underlying system it broke or iterate through all the tests and fix them.

Doing this iteratively has made the overall process for an app I'm trying to implement 100% using LLMs to take at least 3x longer than I would have built it myself. That said, it's unclear I would have kept building this app without using these tools. The process has kept me in the game - so there's definitely some value there that offsets the longer implementation time.

"People use AI to do the same tasks with less effort" maps onto what we've seen with other types of workplace automation - like Excel formulas or VBA scripts.

Why report to your boss that you managed to get a script to do 80% of your work, when you can just use that script quietly, and get 100% of your wage with 20% of the effort?

That aligns well with past ideas, but it doesn't align with the studies that have been performed, where there aren't any of the conflicting priorities you mention.

>1. AI doesn't improve productivity and people just have cognitive biases. (logical, but I also don't think it's true from what I know...)

It is from what Ive seen. It has the same visible effect on devs as a slot machine giving out coins when it spits out something correct. Their faces light up with delight when it finally nails something.

This would explain the study that showed a 20% decline in actual productivity where people "felt" 20% more productive.

(1) seems very plausible, if only because that is what happens with ~everything which promises to improve productivity. People are really bad at self-evaluating how productive they are, and productivity is really pretty hard to externally measure.

(3) and (4) are likely true.

In theory competition is supposed to address this.

However, our evaluation processes generally occur on human and predictable timelines, which is quite slow compared to this impulse function.

There was a theory that inter firm competition could speed this clock up, but that doesn't seem plausible currently.

Almost certainly AI will be used, extensively, for reviews going forward. Perhaps that will accelerate the clock rate.

6. It’s now easier to get something off the ground but structural debt accumulates invisibly. The inevitable cleanup operation happens outside of the initial assessed productivity window. If you expand the window across time and team boundaries the measured productivity reverts to the mean.

This options is insidious in that not only people initially asked about the effect are initially oblivious, it is very beneficial for them to deny the outcome altogether. Individual integrity may or may not overcome this.

2,3,4. While my agent refactors code, I do housework: fold laundry, wash dishes, stack firewood, prep food, paint the deck. I love this new life of offering occasional advice, then walking around and using my hands.

What's the difference between 1 & 5?

I've personally witnessed every one of these, but those two seem like different ways to say the same thing. I would fully agree if one of them specified a negative impact to productivity, and the other was net neutral but artificially felt like a gain.

It's not even claiming that. It's only claiming that people who responded to the survey feel more productive. (Unless you assume that people taking this survey have an objective measure for their own productivity.)

> Significant productivity gains: Over 80% of respondents indicate that AI has enhanced their productivity.

_Feeling_ more productive is inline with the one proper study I've seen.

Well I feel and I am more productive, now on coding activities, I am not convinced, it basically replaced SO and google, but at the end of the day, I always need and want to check reference material that I may have known or not existed. Plenty of time, Google couldn't even find them.

So in my case, yes but not on activities these sellers are usually claiming.

The METR study showed even though people feel more productive they weren’t https://arxiv.org/abs/2507.09089

the MTR study is a joke. it surveyed only 16 devs. in the era of Sonnet 3.5

Can we stop citing this study

I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently (between June 13 and July 21, 2025) which means using the most recent SOTA models

Yea cite the study funded by a company that invested billions into AI instead, that will surely be non biased and accurate.

It didn't "survey" devs. It paid them to complete real tasks while they were randomly assigned to use AI or not, and measured the actual time taken to complete the tasks vs. just the perception. It is much higher quality evidence than a convenience sample of developers who just report their perceptions.

> I'm not saying the DORA study is more accurate, but at least it surveyed 5000 developers, globally and more recently

It's asking a completely different question; it is a survey of peoples' _perceptions of their own productivity_. That's basically useless; people are notoriously bad at self-evaluating things like that.

This seems to be a poll of _users_. "Do people think it has improved _their_ productivity?" is a very different question to "Has it empirically improved aggregate productivity of a team/company/industry." People think _all_ sorts of snake oil improve their productivity; you can't trust people to self-report on things like this.

Reply on news.ycombinator.com