idk. I get more stuff done, faster, that I wouldn’t have been able to. You have to use it judiciously and with supervision - I can see how letting it get out of control would negate the benefits. But with good use, its undeniable.
That might be because of some lag, but another explanation might be that the tools are really effective at increasing someone's perception of their productivity while net productivity gains measured over long time windows are low or zero. Another explanation might be that the incompetent/negligent users are causing enough harm to nearly neutralize the gains brought by proficient / effective users. I don't know what the explanation is, but there is a mystery in the divergence between users' perceptions of their own productivity, and the observed productivity gains in the economy.
e.g. one explanation might be that time spent prompting is time spent not improving, or learning, so it would not be surprising then that the productivity gains might be similar in scale to the productivity lost from learning / improvement that never happens.
another explanation might be that prompting just delays work that will have to be done later when fixing errors. When encountering an error that the LLM doesn't fix on the first attempt the human-in-the-loop doesn't understand the problem very well: they will either pull the slot machine for a while until they hit lucky, or grit their teeth and sit down to understand what the code is doing in the excruciating and boring detail required to find and solve bugs, which is exactly the work that they had offloaded onto the LLM earlier.
Do coding agents help me with highly technical work where it’s some deep thought and discussion with coworkers and eventually I change five lines of code? Not really.
But I can debug a weird kernel panic or optimizer bug from a standing start in an hour when it would have taken me all day without these tools.
As long as you make sure you genuinely understand everything that comes out of it, and your domain has enough coverage in the LLM training data/context, its an incredible tool.
I’m in the final prep for ship phase of a project that started February. It’s a bottom-to-top rewrite of a project that’s been shipping for two years, and took two years to write, initially.
I’m deliberately doing it all with the $20/month ChatGPT sub. After it ships, I’ll move to the $100/month sub; but I want to be able to say that the entire project was done on a low-tier LLM subscription.