Okay.
Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load.
My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.
Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load.
My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
People vibe code because they have no clue about any of that. Not because they're slow typers.
In the case of an LLM generated demo, usually deceptively so.
> and give an impression that the design decision has been made.
In the case of a vibed design, this is the opposite of useful for the team.
This isn't exactly novel territory, here, Simon. Let's not pretend I'm asking for something strange, unprecedented, or unreasonable.
I have similar charts across my three main open source projects:
https://github.com/simonw/datasette/graphs/code-frequency
https://github.com/simonw/llm/graphs/code-frequency
https://github.com/simonw/sqlite-utils/graphs/code-frequency
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
I don't know about easy. It's a hard earned lesson of software engineering, backed by research (some of it cited in TFA). Also, there's a related lesson: the more LoC to solve a given problem, the more bugs. That's also a finding backed by research.
I'll be reading your article when you write it, but I confess I'm skeptical.
The AI companies have a vested interest in using this metric, since it's easy to measure (the reason LoC were used even before AI) and there's no doubt that LLMs are writing tons of code. This makes me doubly skeptical.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
Really now?
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
I ask for evidence.
In response you give me... code output metrics?
That's why Microsoft did a study (referenced in the article), where they measured the time spent on things, so they could get to the truth of how much time people spend coding using "a shape of evidence would you find convincing"
I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs
I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome
So yes "literally everyone has access to it" (what a revelation) but you still need to be smart in how you use it