Where's the shovelware? Why AI coding claims don't add up
mikelovesrobots.substack.com
mikelovesrobots.substack.com
This is my biggest problem right now. The types of problems I'm trying to solve at work require careful planning and execution, and AI has not been helpful for it in the slightest. My manager told me that the time to deliver my latest project was cut to 20% of the original estimate because we are "an AI-first company". The mass hysteria among SVPs and PMs is absolutely insane right now, I've never seen anything like it.
Lord, forgive them, they know not what they do.
"Bobby Lehman is ninety three years old and he dances the twist. He is 100 years old! 120! Maybe 140! He dances like a madman!"
"A bunch of mindless jerks who'll be the first against the wall when the revolution comes."
The thing about hype cycles (including AI) is that the marketing department manages to convince the purchases to do their job for them.
You see yourselves as the disenfranchised proletariats of tech, crusading righteously against AI companies and myopic, trend-chasing managers, resentful of their apparent success at replacing your hard-earned skill with an API call.
It’s an emotional argument, born of tribalism. I’d find it easier to believe many claims on this site that AI is all a big scam and such if it weren’t so obvious that this underlies your very motivated reasoning. It is a big mirage of angst that causes people on here to clamor with perfunctory praise around every blog post claiming that AI companies are unprofitable, AI is useless, etc.
Think about why you believe the things you believe. Are you motivated by reason, or resentment?
The two types of responses to AI I see are your very defensive type, and people saying "I don't get it".
Their claim following that is that because there hasn't been an exponential growth in App store releases, domain name registrations or Steam games, that, beyond just AI producing shoddy code, AI has led to no increase in the amount of software at all, or none that could be called remarkable or even notable in proportion to the claims made by those at AI companies.
I think this ignores the obvious signs of growth in AI companies which providing software engineering and adjacent services via AI. These companies' revenues aren't emerging from nothing. People aren't paying them billions unless there is value in the product.
These trends include
1. The rapid growth of revenue of AI model companies, OpenAI, Anthropic, etc. 2. The massive growth in revenue of companies that use AI including Cursor, replit, loveable etc 3. The massive valuation of these companies
Anecdotally, with AI I can make shovelware apps very easily, spin them up effortlessly and fix issues I don't have the expertise or time to do myself. I don't know why the author of TFA claims that he can't make a bunch of one-off apps with capabilities avaliable today when it's clear that many many people can, have done so, have documented doing so, have made money selling those apps, etc.
You can't use growth of AI companies as evidence to refute the article. The premise is that it's a bubble. The growth IS the bubble, according to the claim.
> I don't know why the author of TFA claims that he can't make a bunch of one-off apps
I agree... One-off apps seem like a place where AI can do OK. Not that I care about it. I want AI that can build and maintain my enterprise B2B app just as well as I can in a fraction of the time, and that's not what has been delivered.
> I want AI that can build and maintain my enterprise B2B app just as well as I can in a fraction of the time, and that's not what has been delivered.
AI isn't at that level yet but it is making fast strides in subsets of it. I can't imagine systems of models and the models themselves won't reach there in a couple years given how bad AI coding tools were just a couple years ago.
Oh, of course not. Just like people weren't paying vast sums of money for beanie babies and dotcoms in the late 1990s and mortgage CDOs in the late 2000s [EDIT] unless there was value in the product.
People paid a lot for beanie babies and various speculative securities on the assumption that they could be sold for more in the future. They were assets people aimed to resell at a profit. They had no value by themselves.
The source of revenue for AI companies has inherent value but is not a resell-able asset. You can't resell API calls you buy from an AI company at some indefinite later date. There is no "market" for reselling anything you purchase from a company that offers use of a web app and API calls.
I think the article's premise is basically correct - if we had a 10x explosion of productivity where is the evidence? I would think some is potentially hidden in corporate / internal apps but despite everyone at my current employer using these tools we don't seem to be going any faster.
I will admit that my initial thoughts on Copilot were that "yes this is faster" but that was back when I was only using it for rote / boilerplate work. I've not had a lot of success trying to get it to do higher level work and that's also the experience of my co-workers.
I can certainly see why a particular subset of programmers find the tools particularly compelling, if their job was producing boilerplate then AI is perfect.
The fundamental difference of opinion people have here though is some people see current AI capabilities as a floor, while others see it as a ceiling. I’d agree with arguments that AI companies are overvalued if current models are as capable as AI will ever be for the rest of time, but clearly that is not the case, and very likely, as they have been every few months over the past few years, they will keep getting better.
It's not ONE person. I agree that it's not "every single human being" either, but more of a preliminary result, but I don't understand why you discount results you dislike. I thought you were completely rational?
https://www.theregister.com/2025/07/11/ai_code_tools_slow_do...
Mousing implies things are visible and you merely point to them. Keyboard implies things are non-visible and you recall commands from memory. These two must have a principal difference. Many animals use tools: inanimate objects lying around that can be employed for some gain. Yet no animal makes a tool. Making a tool is different from using it because to make a tool one must foresee the need for it. And this implies a mental model of the world and the future, i.e. a very big change compared to simply using a suitable object on the spot. (The simplest "making" could be just carrying an object when there is no immediate need for it, e.g. a sufficiently long distance. Looks very simple and I myself do not know if any animals exhibit such behavior, it seems to be on the fence. It would be telling if they don't.)
I think the difference between mousing and keying is about as big as of using a tool and making a tool. Of course, if we use the same app all day long, then its keys become motor movements, but this skill remains confined to the app.
Also AI has been basically useless every time I tried it except converting some struct definitions across languages or similar tasks, it seems very unlikely that it would boost productivity by more than 10% let alone 400%.
FWIW, my own experiences with AI have ranged from mediocre to downright abysmal. And, no, I don't know which models the tools were using. I'm rather annoyed that it seems to be impossible to express a negative opinion about the value of AI without having to have a thoroughly documented experiment that inevitably invites the response that obviously some parameter was chosen incorrectly, while the people claiming how good it is get to be all offended when someone asks them to maybe show their work a little bit.
It’s like saying “I drove a car and it was horrible, cars suck” without clarifying what car, the age, the make, how much experience that person had driving, etc. Of course its more difficult to provide specifics than just say it was good or bad, but there is little value in claims that AI is altogether bad when you don’t offer any details about what it is specifically bad at and how.
That's an interesting comparison. That kind of statement can be reasonably inferred to be made by someone just learning to drive who doesn't like the experience of driving. And if I were a motorhead trying to convert that person to like driving, my first questions wouldn't be those questions, trying to interrogate them on their exact scenario to invalidate their results, but instead to question what aspect of driving they don't like to see if I could work out a fix for them that would meaningfully change their experience (and not being a motorhead, the only thing I can think of is maybe automatic versus manual transmission).
> there is little value in claims that AI is altogether bad when you don’t offer any details about what it is specifically bad at and how.
Also, do remember that this holds true when you s/bad/good/g.
But if all we have to go on is "I used it and it sucked" or "I used it and it was great", like, okay, good for you?
"Damn, these relational databases really suck, I don't know why anyone would use them, some of the data by my users had emojis in them and it totally it! Furthermore, I have some bits of data that have about 100-200 columns and the database doesn't work well at all, that's horrible!"
In some cases knowing more details could help, for example in the database example a person historically using MySQL 5.5 could have had a pretty bad experience, in which case telling them to use something more recent or PostgreSQL would have been pretty good.
In other cases, they're literally just holding it wrong, for example trying to use a RDBMS for something where a column store would be a bit better.
Replace the DB example with AI, same principles are at play. It is equally annoying to hear people blaming all of the tools when some are clearly better/worse than others, as well as making broad statements that cannot really be proven or disproven with the given information, as it is people always asking for more details. I honestly believe that all of these AI discussions should be had with as much data present as possible - both the bad and good experiences.
> If your experience makes you believe that certain tools are particularly good--or particularly bad--for the tasks at hand, you can just volunteer those specifics.
My personal experience:
* most self-hosted models kind of suck, use cloud ones unless you can get really beefy hardware (e.g. waste a lot of money on them)
* most free models also aren't very good, nor have that much context space
* some paid models also suck, the likes of Mistral (like what they're doing, just not very good at it), or most mini/flash models
* around Gemini 2.5 Pro and Claude Sonnet 4 they start getting somewhat decent, GPT 5 feels a bit slow and like it "thinks" too much
* regardless of what you do, you still have to babysit them a lot of the time, they might take some of the cognitive load off, but won't make you 10x faster usually, the gains might definitely be there from reduced development friction (esp. when starting new work items)
* regardless of what you do, they will still screw up quite a bit, much like a lot of human devs do out there - having a loop of tests will be pretty much mandatory, e.g. scripts that run the test suite and also the compilation
* agentic tools like RooCode feel like they make them less useless, as do good descriptions of what you want to do - references to existing files and patterns etc., normally throwing some developer documentation and ADRs at them should be enough but most places straight up don't have any of that, so feeding in a bunch of code is a must
* expect usage of around 100-200 USD per month for API calls if the rate limits of regular subscriptions are too limiting
And if they don't, then you'd understand the anger surely. You can't say "well obviously everybody should benefit" and then also scold the people who are mad that everybody isn't benefiting.
I’m not concerned for my job, in fact I’d be very happy if real AGI would be achieved. It would probably be the crowning tech achievement of the human race so far. Not only would I not have to work anymore, the majority of the world wouldn’t have to. We’d suddenly be living in a completely different world.
But I don’t believe that’s where we’re headed. I don’t believe LLMs in their current state can get us there. This is exactly like the web3 hype when the blockchain was the new hip tech on the block. We invent something moderately useful, with niche applications and grifters find a way to sell it to non technical people for major profit. It’s a bubble and anyone who spends enough time in the space knows that.
I agree that there are lots of limitations to current LLM's, but it seems somewhat naive to ignore the rapid pace of improvement over the last 5 years, the emergent properties of AI at scale, especially in doing things claimed to be impossible only years prior (remember when people said LLM's could never do math, or that image models could never get hands or text right?).
Nobody understands with greater clarity or specificity the limitations of current LLM's than the people working in labs right now to make them better. The AGI prognostications aren't suppositions pulled out of the realm of wishful thinking, they exist because of fundamental revelations that have occurred in the development of AI as it has scaled up over the past decade.
I know I claimed that HN's hatred of AI was an emotional one, but there is an element to their reasoning too that leads them down the wrong path. By seeing more flaws than the average person in these AI systems, and seeing the tact with which companies describe their AI offerings to make them seem more impressive (currently) than they are, you extrapolate that sense of "figuring things out" to a robust model of how AI is and must really be. In doing so, you pattern match AI hype to web3 hype and assume that since the hype is similar in certain ways, that it must also be a bubble/scam just waiting to pop and all the lies are revealed. This is the same pattern-matching trap that people accuse AI of making, and see through the flaws of an LLM output while it claims to have solved a problem correctly.
And that´s actually quite useful - given that most of this material is paywalled or blocked from search engines. It´s less useful when you look at code examples that mix different versions of python, and have comments referring to figures on the previous page. I´m afraid it becomes very obvious when you look under the hood at the training sets themselves, just how this is all being achieved.
All intelligence is pattern matching, just at different scales. AI is doing the same thing human brains do.
Hard not to respond to that sarcastically. If you take the time to learn anything about neuroscience you'll realise what a profoundly ignorant statement it is.
But even if it's not a lot, it's more than the number of LLMs that can invent new meaning which is a grand total of 0.
If tomorrow, all LLMs ceased to exist, humans would carry on just fine, and likely build LLMs all over again, next time even better.
LLMs are not anything like Web3, not "exactly like". Web3 is in no way whatsoever "something moderately useful", and if you ever thought it was, you were fooled by the same grifters when they were yapping about Web3, who have now switched to yapping about LLMs.
The fact that those exact same grifters who fooled you about Web3 have moved onto AI has nothing to do with how useful what they're yapping about actually is. Do you actually think those same people wouldn't be yapping about AI if there was something to it? Yappers gonna yap.
But Web3 is 100% useless bullshit, and AI isn't: they're not "exactly alike".
Please don't make false equivalences between them like claiming they're "exactly like" each other, or parrot the grifters by calling Web3 "moderately useful".
I think most people are motivated by values. Reason and emotion are merely tools one can use in service of those.
My experience that people who hew too strongly to the former tend to be more oblivious to what's going on in their psychology than most.
But I also really care about the quality of our code, and so far my experiments with AI have been disappointing. The empirical results described in this article ring true to me.
AI definitely has some utility, just as the last "game changer" - blockchain - does. But both technologies have been massively oversold, and there will be many, many tears before bedtime.
Please, enlighten me with your gigantic hyper-rational brain.
AI stans don’t become AI stans for no reason. They see the many enormous technological leaps and also see where progress is going. The many PhDs currently making millions at labs also have the right idea.
Just look at ChatGPT’s growth alone. No product in history compares, and it’s not an accident.
Bad framing and worse argument. It's emotional.
Every engineer here is evaluating what ai claims it can do as pronounced by ceos and managers (not expert in software dev) v reality. Follow the money.
Yeah, it's frustrating to see someone opine "critics are motivated by resentment rather than facts" as if it were street-smart savvy psychoanalysis... while completely ignoring how many influential voices boosting the concept have a bajillions of dollars in motive to speak as credulously and optimistically as possible.
I'd widen the frame a bit. People scared of losing their jobs might underestimate the usefulness of AI. Makes sense to me, it's the comforting belief. Worth keeping in mind while reading articles sceptical of AI.
But there's another side to this conversation: the people whose writing is pro AI. What's motivating them? What's worth keeping in mind while reading that writing?
But I am not claiming that AI is useless. It is useful, but I would rather destroy every data center that enjoy strengthening of techno-feudalism.
Yeah so the thing is the "success" is only "apparent". Having actually tried to use this garbage to do work, as someone who has been deeply interested in ML for decades, I've found the tools to be approximately useless. The "apparent success" is not due to any utility, it's due entirely to marketing.
I don't fear I'm missing out on anything. I've tried it, it didn't work. So why are my bosses a half dozen rungs up on the corporate ladder losing their entire minds over it? It's insanity. Delusional.
Damn, when did it become wrong for me to advocate in my best interests while my boss is trying to do the same by shoving broken and useless AI tools up my ass?
If the self checkout scanner at the supermarket started bickering with me for entering the wrong produce code, that would wrap up the whole Turing Test thing for me.
---
[1] We generally budget about half an intern's time for finding the coffee machine, learning how to show up to work on time, going on a fun event with the other interns to play minigolf, discovering that unit tests exist, etc, etc.
I have backend development background so I was able to review the BE code and fix some bugs. But I did not bother learning Jira and Harvest API specs at all, AI (cursor+sonnet 4) figured it out all.
I would not be able to write the front-end of this. It is JS based and updates the UI based on real-time http requests (forgot the name of this technology, the new ajax that is) and I do not have time to learn it but again, I was able to tweak what AI generated and make it work.
Not only AI helped me do something in much shorter than it would take, it enabled me do something that otherwise would not be possible.
If we can delegate incident response to automated LLMs too, sure, why not. Let the CEO have his way and pay the reputational price. When it doesn't work, we can revert our git repos to the day LLMs didn't write all the code.
I'm only being 90% facetious.
I think making stakeholders have to engage with these models is the most critical point for people having deadlines or expectations based on them.
Let Claude run incident response for a few weeks. I'll gladly pause pagerduty for myself.
Challenge your manager to a race, have him vibe code
That's insane. Who the hell pulls a number out of their ass and declares it the new reality? When it doesn't happen, he'll pin the blame on you, but everyone else above will pin the blame on him. He's the one who will get fired.
Laying off unnecessary developers is the answer if LLMs turn out to make us all so much more productive (assuming we don't just increase the amount of software written instead). But that happens after successful implementation of LLMs into the development process, not in advance.
Starting to think I should do the inadvisable and move my investments far far away from the S&P 500 and into something that will survive the hype crash that can't be too far off now.
Not investing advice; the bottom 490 companies in the S&P500 are nominally flat since 2022 and down against inflation, GPUs and AI hype are holding everything together at the moment.
> In simpler terms, 35% of the US stock market is held up by five or six companies buying GPUs. If NVIDIA's growth story stumbles, it will reverberate through the rest of the Magnificent 7, making them rely on their own AI trade stories.
https://www.wheresyoured.at/the-haters-gui/
> Capex spending for AI contributed more to growth in the U.S. economy in the past two quarters than all of consumer spending, says Neil Dutta, head of economic research at Renaissance Macro Research, citing data from the Bureau of Economic Analysis.
https://www.bloodinthemachine.com/p/the-ai-bubble-is-so-big-...
> Two Nvidia customers made up 39% of Nvidia’s revenue in its July quarter, the company revealed in a financial filing on Wednesday, raising concerns about the concentration of the chipmaker’s clientele.
https://www.cnbc.com/2025/08/28/nvidias-top-two-mystery-cust...
Chatgpt.
I wouldn’t count on it.
Product and Sales?
I'd argue that compared to a decade. 15 years ago, relatively little value has been created. If you sat down in front of a 15 yo computer, or tried to solve a technical challenge with the tooling of 15-10 years ago, I don't think you'd get a significantly worse result.
Yet in this time the US has doubled its GDP, most of it owning to the top, to the tech professionals and financiers who benefited from this.
And some of this money went into assets with constrained supplies, such as housing, marking up the prices adjusted for inflation, making average people that much worse off.
While I do feel society is making progress, it's been a slow and steady march. Which in relative terms means slowing. Of I gave you $10 every week, by week 2, you'd double your wealth, by the end of the year, you almost didn't notice the increase.
Technology accumulation is the same, I'd argue it's even worse, since building on top of existing stuff has a cost proportional to the complexity for a fixed unit of gain (and features get proportionally less valuable as you implement the most important ones first).
Sorry got distracted from my main point - what happens when people stop believing that these improvements are meanigful or that technology that was priced in to produce 100x the value will come at all (and more importantly the company you're invested in will be able to caputre it)?
While you have decent points in your comment (essentially, the idea of tech industry growth slowing due to low hanging fruit being picked), if this statement going to be your barometer you’re going to end up looking stupendously wrong.
You can sit your Grandma down at her computer and have her type in “please make me a website for my sewing club that includes a sign up form and has a pink background” and AI will just do it and it’ll probably even work the first time you run it.
15 years ago tossing a website on Heroku was a pretty new concept, and you definitely had to write it all on your own.
10 years ago Kubernetes had its initial release.
Google Drive and Slack are not even 15 years old.
TensorFlow is just hitting its 10th birthday.
I think you’re vastly underestimating the last 15 years of technological progress and in turn value creation.
25 years ago we had wysiwyg editors to build web pages, and she could just include a link to email her to have people ask her to "sign up" (the entire point of a sewing club is to socialize. You don't need an automated newsletter or whatever. That's for people that want to sell you something). You'd put it into the space your ISP gave you, or Windows actually included a personal web server. You had to be somewhat technical to use the Windows server, but it could've been made more friendly instead of being removed, and personal computers should make it easy to share a static website.
We've clearly regressed since then. People think you need an engineer or AI to build basic web pages. My actual grandma was doing some of this stuff when I was a kid in the 90s, as was I.
What happened to the web seems analogous to me to as if we abandoned word processors and spreadsheets and said you need a six-figure engineer to do custom programming any time you need the features of one of those things.
You should not look back on those solutions and neglect to acknowledge their limitations. Access can only handle small databases (maximum 2GB) and about 100 simultaneous connections.
Access is basically a product of the technological limitations of its time. It isn’t designed the way it is because that way is the best design.
The kind of business that relied on Access is going to find that solutions like Airtable are far easier and more powerful, and certainly less prone to data loss than mom and pop’s shared drive. There are even open source self-hosted alternatives like NocoDB and Baserow (or cloud hosted).
You’ll inevitably complain about the subscription cost of SaaS solutions but it’s not like MS Office business licenses were ever cheap.
But when it comes to actually accomplishing something with application logic on the web, Grandma could ask the AI agent literally all these questions and get solid answers. At any point you get confused you can just ask what to do. You can highlight a line of code and ask “what does this mean?” You can even say “I want this web page to be put online what do I do?”
Beats waiting for a human to answer you.
You’re also taking my example application way too literally. No I don’t know why Grandma needs a signup form, I just couldn’t think of a web app off the top of my head.
MS Access and WSYWIG tools like FrontPage and iWeb were not good. I know because I was there and I used FrontPage at work. A link to your email (get spammed forever) is not a replacement for an application or email form. The whole reason code is preferred over WYSIWYG is because you inevitably have issues with change management even for simple personal projects, which is why static site generators have gained popularity. I’m sure your grandma could have handled markdown + someone else’s Hugo template + Netlify.
Hell, if we want to talk about progress in WYSIWYG editors, Trix greatly improved and modernized the shortfalls of that process and that was launched in 2014, less than 10 years ago. So even in the world of WSYWIG we have better tools now than before.
IIS has not been removed from Windows home edition, by the way.
Apart from doing the styles and layout, I don't think current tools have less friction. They're a lot safer though. Can't say I never dropped a production database.
I definitely agree that it’s a reasonable choice.
As for grandma, 15 years ago she could have just posted her sewing club on Facbook, she doesn't need Heroku or AI.
I would say that these technologies/products being so wildly popular puts burden of proof on you to show me some kind of evidence that these technologies aren’t productive. Like are you trying to say that something had better deployment velocity and reliablity than Kubernetes at handling high-complexity application infrastructure deployments for large enterprise companies? What was it? Why was it better than Kubernetes?
The analogy is that you’re basically saying that zippers aren’t really better than buttons but then literally everyone is overwhelmingly wearing pants and coats with zippers and very strongly prefer zippers. So really it’s on you to prove to me that I should be using pants with buttons instead.
Finally, there’s a lot of irony in your first paragraph complaining about a few tech oligarchs becoming fabulously wealthy and then suggesting that Grandma just use Facebook instead of building her own site. In any event, my web app example was just a poorly thought out example of a web app, I really just mean a website that has a little more utility than a static site.
The juice hasn't been worth the squeeze. You can look at all societal indicators except the stock market pointing downward to get to that conclusion. Nothing is actually better than it was in 2010 despite Uber, Airbnb, Kubernetes, Slack, and all the other SV tech "innovations". People are not happier or wealthier because of the tech coming from Silicon Valley. In general the end result of the last 15 years of tech is that it's made us more neurotic, disconnected, depressed, and angry.
We don't need "better deployment velocity and reliability for high-complexity application infrastructure deployments for large enterprises". Listen to yourself man, you sound like you've been hypnotized by the pointy-haired boss. The tech sector makes false promises about a utopia future, and then it just delivers wealth for shareholders, leaving everyone else worse off.
Grandma especially doesn't need deployment velocity, she's being evicted by her landlord because he wants to turn her flat into an Airbnb. She can't get to the grocery store because the town won't invest in public transport and Uber is the only option. She's been radicalized by Meta and Youtube and now she hates her own trans grandchild because her social media feed keeps her algorithmically outraged. Oh, and now she's getting scammed by AI asking her to invest her life savings in shitcoins and NFTs.
> The analogy is that you’re basically saying that zippers aren’t really better than buttons but then literally everyone is overwhelmingly wearing pants and coats with zippers and very strongly prefer zippers.
I don't agree that the ubiquity and utility are necessarily correlated, so I don't see the zippers and Kubernetes as analogous.
But the proliferation of zippers has more to do with the fact they are easier for manufacturers to integrate into products compared to buttons -- they come pre-measured and installing them is a straight stitch that can be done with a machine, whereas installing buttons is more time-consuming.
Zippers are worse for consumers in many ways, repairability chief among them. But really they are part of a general trend over my lifetime of steadily falling garment quality, as manufacturers race to the bottom.
> In any event, my web app example was just a poorly thought out example of a web app, I really just mean a website that has a little more utility than a static site.
You said it, not me. We had the technology to throw up a static site in 2010 and my grandmother could actually do that with dreamweaver and FTP, and it worked fine.
2. Like it or not, the customers of the tech industry include a lot of large enterprises that do find value in improving velocity and reliability for complex workflows. I am not some kind of corporate sellout for pointing out this plain factual reality.
3. I totally agree with your points about income inequality and happiness metrics among our population but they are not relevant to the topic at hand.
4. If Dreamweaver and FTP is your barometer, recall that Dreamweaver was an expensive paid product. There still are plenty of FTP-based web hosts and you can totally throw up a website with a tool like https://trix-editor.org/, which as I mentioned is a tool that did not exist 15 years ago. You can also just pay for a website service like SquareSpace or Wix (or not, they have free tiers).
The fact that specific individual tools do not exist/are not supported anymore is irrelevant, as there are dozens of often-better tools for throwing a website up that are definitely friendly to novices. Let's not forget the plethora of no-code application development tools that are mostly a recent development.
Mobile users prefer apps 60% over websites. So the real modern barometer is: could your Grandma put up an iPhone app in 2008 following the developer tutorials using Objective-C or would she have a much better time using Swift and/or a no-code app development solution?
You're assuming that people wear pants with zippers because they have a preference for it, and not because you literally cannot buy pants with buttons in 99% of stores.
based on your examples, I'd say you're vastly overestimating
sure, those are all new techs, but it's not like you couldn't do those things before in different ways or they are revolutionary in any way
In case you also need to control Spotify from Windows 95 :D
And to me this is worse news. People in higher paying jobs are the ones that would hurt the economic fabric more, but by that token they’d have more power and influence to ensure a better safety net for the inevitable rise of AI and automation in much of the workforce.
Entry level workers can’t afford to not work, they can’t afford to protest or advocate, they can’t afford the future that AI is bringing closer to their doorsteps. Without that safety net, they’ll be struggling and impoverished. And then will everyone in the higher paying positions help, or will we ignore the problem until AI actually is capable of replacing us, and will it be too late by then?
if history is anything to go by, it'll be the latter, sadly
This isn't entirely foreign to me; it sure looks a lot like the hype train of the dot-com bubble. My experience says that if you're holding stocks in a company going down this road, I'd say they have very low long-term value. Even if you think there's room to grow, bubbles pop fast and hard.
It’s that we are heading towards a big recession.
As in all recessions, people come up with all sorts of reasons why everything is fine until it can’t be denied anymore. This time, AI was a useful narrative to have lying around.
2030 will be 2020 all over again.
When I started a CS degree in 2003, we were still kinda in the "dot com crash has happened, no-one will ever hire a programmer again" phase, and there were about 50 people in my starting class. I think in the same course two years ago, there'd been about 200. The 'correct' number, for actual future demand, was certainly closer to 200 than 50, and the industry as a whole had a bit of a labour crunch in the early 10s in particular.
This means that they inflate programmer salaries, which makes it impossible for most companies that could benefit from software development to hire developers.
We could probably have five times as many software developers as we have now, and they would not be out of work; they would only decrease average salaries for programmers.
Why would this be the case? Many programmers join Google or Meta (or similar tier companies) and immediately double or triple their income. Software salaries are famously bimodal and people often transition from the lower mode to the higher mode practically overnight.
In fact (and I'm not an economist) I conjecture that the lower mode exists because the upper mode exists. That is, people purposefully don't really care what their salary is (i.e. don't put upward wage pressure) when they're at lower-mode companies because they know one day they'll make the leap to the upper-mode. In other words, the fact that Google-tier companies pay well allows other companies to pay poorly because those guys are just padding their resumes to get a 350k job at Google and don't really care whether Bank of Nowhere pays them $90k or $110k.
1) fewer students are studying computer science, I'm faculty at a top CS program and we saw our enrollment decline for the first time ever. Other universities are seeing similar slowdowns of enrollment [1]
2) fewer immigrants coming to the united states to work and live, US is perhaps looking at its first population decline ever [2]
3) Current juniors are being stunted by AI, they will not develop the necessary skills to become seniors.
4) Seniors retiring faster because they don't want to have to deal with this AI crap, taking their knowledge with them.
So we're looking at a negative bubble forming in the software engineering expertise pipeline. The money people are hoping that AI can become proficient enough to fill that space before before everything bursts. Engineers, per usual, are pointing out the problem before it becomes one and no one is listening.
[1]: https://www.theatlantic.com/economy/archive/2025/06/computer...
[2]: https://nypost.com/2025/09/03/us-news/us-population-could-sh...
2. It looks like rates will keep going down.
3. Fewer people are going into CS due to the AI hysteria. You might say oh there's a 4 year lag, but not quite. We should see an immediate impact from career changers, CS grads choosing between career and professional school, and those switching out of CS careers.
The tech AI fear hysteria is so widespread that I've even heard of people avoiding non-SWE tech careers like PM.
Who is we? One country heading into a recession is hardly enough to nudge the trend of "all things code"
I wonder how that would even be measured? I suppose you could do it for roles that do the same type of work every day. I.e. perhaps there is some statistical relevance to number of calls taken in a call center per day or something like that. One the software development side however, productivity metrics are very hard to quantify. Of course, you can make a dashboard look however you want, but impossible, essentially to tie those metrics to NPV.
I'm legit not sure if that's sarcasm or not
And my answer - like the best of car mechanics who work on custom rides - is: I don't know how long until I actually get in there, but minimum 5x what you think, and my rate is $300/hr. Is it worth it to you to do it right?
Usually the answer is no. When it's yes, I have a free hand. And having a few clients who pay well is worth a lot more than having a few dozen who think they know everything and are too cheap to pay for it anyway.
And regex.
Part of the issue is that it is so fast to get to a working prototype that it feels like you are almost there, but there is a lot of effort to go.
People made tools specifically to make templates look like templates and not a finished product, just because client was ready to push the product just after seeing real looking mockups of interface.
Not too different in that respect from non-AI-assisted coding! The "last 10%" is always more like 50-80%.
That's pretty much nothing ... to me that line indicates a whole lot of other possible things.
Tell him to code it himself then? If it can be done with only prompting, and he's able to type a sentence or two in a web form, what's stopping him?
This sounds incredibly stupid. It’s going to take as long as it will and if they’re not okay with that, their delusional estimates should be allowed to crash and burn, which would hopefully be a learning experience.
The problem is that sometimes there’s an industry wide hysteria even towards useful tech - like doing a lift and shift of a bunch of monoliths to AWS to be “cloud scale”, introducing Kubernetes or serverless without the ability to put either to good use, NoSQL for use cases it’s not good at and most recently AI.
I think LLMs will eventually weather the hype cycle and it will settle down on what they’re actually kinda okay at vs not, the question is how many livelihoods will be destroyed along the way (alongside all the issues with large scale AI datacenter deployments).
On a personal level, it feels like you should maybe do the less ethical thing of asking your employer in the ballpark of 1000-3000 USD for Claude credits a month, babysitting it enough to do the 20% of the work in the 20% new estimate, babysit it enough to ship a functional MVP and when they complain about missing functionality tell them that the AI tech just isn't mature enough but thankfully you'll be able to swoop in and salvage it for only the remaining 80% of the remaining estimate's worth of work.
1. LLMs do not increase general developer productivity by 10x across the board for general purpose tasks selected at random.
2. LLMs dramatically increases productivity for a limited subset of tasks
3. LLMs can be automated to do busy work and although they may take longer in terms of clock time than a human, the work is effectively done in the background.
LLMs can get me up to speed on new APIs and libraries far faster than I can myself, a gigantic speedup. If I need to write a small bit of glue code in a language I do not know, LLMs not only save me time, but they make it so I don't have to learn something that I'll likely never use again.
Fixing up existing large code bases? Productivity is at best a wash.
Setting up a scaffolding for a new website? LLMs are amazing at it.
Writing mocks for classes? LLMs know the details of using mock libraries really well and can get it done far faster than I can, especially since writing complex mocks is something I do a couple times a year and completely forget how to do in-between the rare times I am doing it.
Navigating a new code base? LLMs are ~70% great at this. If you've ever opened up an over-engineered WTF project, just finding where HTTP routes are defined at can be a problem. "Yo, Claude, where are the route endpoints in this project defined at? Where do the dependency injected functions for auth live?"
Right tool, right job. Stop using a hammer on nails.
So amazing that every single stat showed by the author in the article has been flat at best, despite all being based on new development rather than work on existing code-bases.
I wax and wane on this one.
I've had the same feelings, but too often I've peaked behind the curtain, read the docs and got familiar with external dependencies and then realize whatever the LLM responds with paradoxically either wasn't following convention or tried to shoehorn your problem to fit code examples found online, used features inappropriately, took a long roundabout path to do something that can be done simply, etc.
It can feel like magic until you look too closely at it, and I worry that it'll make me complacent with the feeling of understanding without actually taking away an understanding.
If I have to manually verify every answer, I may as well read the docs myself.
You pay money, have vendor lock-in, get one answer, and there's no upvotes/downvotes/accepted-answers/moderation or clarification.
It's incredible how quickly an LLM can answer. I've also crossed checked its responses with documentation before and discovered that it suggested implementing a deprecated feature that had a massive warning banner in the documentation that the LLM failed to indicate. I'm still a fan of reading documentation.
The difference is that if I go directly to the support site, there's a decent chance I can quickly spot and reject the garbage based on the date, the votes it's gotten, even the quality of the writing. AI doesn't include any of those clues; it mixes good and bad together and offers it up for me to pick apart through trial and error.
For questions that I know should have a straightforward answer, I think it beats searching Stackoverflow. Sure, I'll typically end up having to rewrite most of the script from scratch; however, if I give it a crude starting point of a half-functional script I've already got going, pairing that with very clear instructions on how I'd like it extended is usually enough to get it to write a proof of concept demonstration that contains enough insightful suggestions for me to spend some time reading about features in man pages I hadn't yet thought to use.
The biggest problem maybe is a propensity for these models to stick in every last fancy feature under the sun. It's fun to read about a GNU extension to awk that makes my script a couple lines shorter, but at best I'll take this as an educational aside than something I'd accept at the expense of portability.
sorry, what am I supposed to use on nails?
What is this supposed busy work that can be done in the background unsupervised?
I think it's about time for the AI pushers to be absolutely clear about the actual specific tasks they are having success with. We're all getting a bit tired of the vagueness and hand waving.
I don't think the original comment you responded to made this specific point.
> LLMs can get me up to speed on new APIs and libraries far faster than I can myself
To this?
> LLMs can get me up to speed on old APIs and old libraries that are new to me far faster than I can myself
My experience is if the library/API/tool is new then the LLM can't help. But maybe I'm using it wrong.
Weren't the code generators before this even better though? They generated consistent results and were dead quick at doing it.
It didn’t work. I asked a colleague. He had the same problem. Turned out it was using out of date setup instructions for a major tool that has changed post training.
After spending time fixing the problem, I realised (1) it would have been faster to do it myself and (2) I can no longer trust that tool to set anything up - what if it’s doing something else wrong?
Use MCP servers, specifically context 7.
This gets up to date docs as long as you include the library name on your prompt and ask to use context 7.
You did the equivalent of raw dogging gpt4(an old model) for recent news versus using an agent with web search tooling.
React had this issue when they changed away from create react app. Basically every tutorial for years was instantly made obsolete, despite occupying all the top search results slots.
Forums where filled with confusion for quite awhile.
Now days most libraries seem to rely on "content producers" to document how to actually do things and don't bother with great docs filled with examples. In decades gone past, companies would hire actual authors to write real physical books on how to use new APIs.
LLMs just have a higher latency on updating their reference material.
You’re right, and in the past I have. So I know to always check the authorities sources for setup instructions.
Traditional documentation has always been a challenge for me - figuring out where to start, what syntax conventions are being used, how pieces connect together. Good docs are notoriously hard to write, and even harder to navigate. But now, being able to query an LLM about specific tasks and get direct references to the relevant documentation sections has been a game-changer.
This realization led me to flip my approach entirely. I’ve started heavily documenting my own development process in markdown files - not for humans, but specifically for LLMs to consume. The key insight is thinking of LLMs as amnesiac junior engineers: they’re capable, but they need to be taught what to do every single time. Success comes from getting the right context into them.
Learning how to craft that context is becoming the critical skill.
It’s not about prompting tricks - it’s about building systematic ways to feed LLMs the information they need.
I’ve built up a library of commands and agents for my Claude Code installation inspired by AgentOS (https://github.com/buildermethods/agent-os) to help engineer the required context.
The tool is a stochastic parrot, you need to feed it the right context to get the right answer. It is very good at what it does but you need to use it to its strengths in order to get value from it.
I find people complaining about LLMs often expect vibe coding to be this magic tool that will build the app for you without thinking, which it unfortunately has been sold as, but the reality is more of a fancy prompt based IDE.
Just a random personal anecdote I wanted to throw out. I recently had to build some custom UI with Qt. I hadn't worked with Qt in a decade and barely remembered it. Seems like a perfect use case for AI to get me "up to speed" on the library, right? It's an incredibly well documented library with lots written on it, perfect fodder for an AI to process.
So, I gave it a good description of the widget I was trying to make, what I needed it to look like and how it should be behave, and behold, it spit out the specific widget subclass I should use and how I should be overriding certain methods to customize behavior. Wow, it worked exactly like promised.
So I implemented it like it suggested and was seemingly happy with the results. Went on with working on other parts of the project, dealing with Qt more and more here and there, gaining more and more experience with Qt over time.
A month or two later, after gaining more experience, I looked back at what AI had told me was the right approach on that widget and realized it was completely messed up. It had me subclassing the completely wrong type of widget. I didn't need to override methods and write code to force it to behave the way I wanted. I could instead just make use of a completely different widget that literally supported everything I needed already. I could just call a couple methods on it to customize it. My new version removes 80% of the code that AI had me write, and is simpler, more idiomatic, and actually makes more sense now.
So yeah, now any time I see people write about how "well, it's good for learning new libraries or new languages", I'll have that in the back of my mind. If you don't already know the library/language, you have zero idea whether what the AI teaching you is horrible or not. Whether there's a "right/better" way or not. You think it's helping you out when really you're likely just writing horrible code.
I do find LLMs useful at times when working in unfamiliar areas, but there are a lot of pitfalls and newly created risks that come with it. I mostly work on large existing code bases and LLMs have very much been a mildly useful tool, still nice to have, but hardly the 100x productivity booster a lot of people are claiming.
This is trivial work that you should have automated after doing it once or twice anyways :/
I think the "why" for this is that the stakes are high. The economy is trembling. Tech jobs are evaporating. There's a high anxiety around AI being a savior, and so, a demi-religion is forming among the crowd that needs AI to be able to replace developers/competency.
That said: I personally have gotten impressive results with AI, but you still need to know what you're doing. Most people don't (beyond the beginner -> intermediate range), and so, it's no surprise that they're flooding social media with exaggerated claims.
If you didn't have a superpower before AI (writing code), then having that superpower as a perceived equalizer is something that you will deploy all resources (material, psychological, etc) to ensuring that everyone else maintain the position that 1) superpower good, 2) superpower cannot go away 3) the superpower being fallible should be ignored.
Like any other hype cycle, these people will flush out, the midpoint will be discovered, and we'll patiently await the next excuse to incinerate billions of dollars.
Which is why they generate so much hype. They are perfect for tech demos, then management wonders why they aren't seeing results in the real world.
For tight tasks it can be super helpful -- like for me, an AI/Data Science guy, setting up a basic reverse proxy. But I do so with a ton of scrutiny -- pushing it, searching on Kagi or docs to at least confirm the code, etc. This is helpful because I don't have a mental map about reverse proxy -- so it can help fill in gaps but only with a lot of reticence.
That type of use really doesn't justify the billion dollar valuations of any companies, IMO.
ie pnpm create vite
Tailwind is similarly a one liner to initialize(might be a vite create option now).
Edit: My bad, you are talking about the LLMs! I'm always surprised how still for past years, even though we have great projects scalfolding across the node verse, people are still complaining about how hard setting up projects is..
You want as much context as possible _right in the code_.
In my experience you don't need to know a whole lot about LLM's to work them. You need to know that everything they spit out is potential garbage, and if you can't tell the good from the garbage then whatever you're using them for is going to be terrible. In terms of software terrible is fine for quite a lot of systems. One of the first things I build out of university in the previous millennium is still in production today and it's horrible. It's inefficient, horribly outdated since it hasn't been updated ever. It runs 10 times a day and at least 1 of them will need to automatically restart itself because it failed. It's done it's job without the need for human intervention for the past many decades though. I know because one of my old colleagues still works there. It could've been improved, but the inefficiency cost over all those years is probably worth about two human hours, and it would likely take quite a while to change it. A lot of software is like that, though a lot of it doesn't live for so long. LLM's can absolutely blast that sort of thing. It's when the inefficiency cost isn't less than a few human hours that LLM's become a liability if you don't know how to do the engineering.
I use LLM's to write a lot of the infrastructure as code we use today. I can do that because I know exactly how that should be engineered. What the LLM can do that I can't, is that it can spit out the k8s yaml for an ingress point with 200 lines of port settings in a couple of seconds. I've yet to have it fail, probably because those configurations are basically all the same depending on the service. What a LLM can't do, however, is write the entire yaml config.
Similarily it can build you a virtual network with subnets in bicep based on a couple of lines of text with address prefixes. At the sametime it couldn't build you a reasonable vnet with subnets if you asked it to do it from scractch. That doesn't mean it can't build you one that works though, it's just that you're likely going to claim 65534 ip addresses for a service which uses three.
On the other hand, I’ve lately seen it misused by less experienced engineers trying to implement bigger features who eagerly accept all it churns out as “good” without realizing the code it produced:
- doesn’t follow our existing style guide and patterns.
- implements some logic from scratch where there certainly is more than one suitable library, making this code we now own.
- is some behemoth of a PR trying to do all the things.
Depending on the amount of code, I see this only as positive? Too often people pull huge libraries for 50 lines of code.
- Implementing a scheduler from scratch (hundreds of lines), when there are many many libraries for this in Go.
- Implementing some complex configuration store that is safe for concurrent access , using generics, reflection, and a whole other host of stuff (additionally hundreds of lines plus more for tests).
While I can't say any of the code is bad, it is effectively like importing a library which your team now owns, but worse in that no one really understands it or supports it.
Lastly, I could find libraries that are well supported, documented, and active for each of these use-cases fairly quickly.
If I as a reviewer don’t know if the author used AI, I can’t even assume a single human (typically the author) has even read any or major parts of the code. I could be the first person reviewing it.
Not that it’s a great assumption to make, but it’s also fair to take a PR and register that the author wrote it, understands it, and considers it ready for production. So much work, outside of tech as well, is built on trust at least in part.
Usually gets them to sort out their behavior without directly making accusations that could be incorrect. If they really did write or strongly review the code, those questions are easy to answer.
Maybe keep your eyes open? :-)
And for the record - my eyes are open. I'm aware I'm being bullshitted. I don't trust, I verify.
But I also don't have a magical lever that I can pull to make it stop hallucinating.
... and every time I ask if one exists, I get either crickets, or a response that doesn't answer the question.
Because, as part of your verification, you will have to do that _anyway_.
I used to be one of those people. It just made sense to me when I was (I still am to some extent) more naïve than I am today. But then I also used to think "it makes sense for everyone to eat together at a community kitchen of some sort instead of cooking at home because it saves everyone time and money" but that's another tangent for another day. The reason I bring it up is I used to think if it is shared functionality and it is a small enough domain, there is no need for everyone to spend time to implement the same idea a hundred times. It will save time and effort if we pool it together into one repository of a small library.
Except reality is never that simple. Just like that community kitchen, if everyone decided to eat the same nutritious meal together, we would definitely save time and money but people don't like living in what is basically an open air prison.
I don't know if this is intended as a joke, if yes this is in very poor taste.
Death cap mushrooms are incredibly dangerous and shouldn't even touch food containers or other food.
There is no safe way to consume death caps. They are the most common cause of human death by mushroom poisoning.
Email validation in 2025 is simple. It has been simple for years now. You check that it contains an '@' with something before it, and something after it. That's all there is to it — then send an email. If that works (user clicks link, or whatever), the address is validated.
This should be well-known by now (HN has a bunch of topics on this, for example). It is something that experienced devs can easily explain too: once this regex lands in your code, you don't want to change it whenever a new unexpected TLD shows up or whatever. Actually implementing the full-blown all edge cases covered regex where all invalid strings are rejected too, is maddeningly complex.
There is no need either; validating email addresses cannot be done by just a regex in any case — either you can send an email there or not, the regex can't tell — and at most you can help the user inputting it by detecting the one thing that is required and which catches most user input errors: it must contain an '@', and something before and after it.
If you try to do what ChatGPT or Copilot suggests you get something more complex:
^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$
> If you need stricter validation or support for internationalized domains (IDNs), I can help you build a more advanced version. Want to see one that handles Unicode or stricter rules?
AI is not helpful here.
suspect this happened because the reimplementation contained a number of standard/expected methods that we didn't have in our existing interface (because we didn't need them), so it was considered 'different' enough. but none of the code actually used those methods (because we didn't need them), so all this PR did was add a few hundred lines of cognitive overhead.
It's difficult because you need team members to be able to work quite independently but knowledge of internal libraries can get so siloed.
I've explored code like FreeBSD, Busybox, Laravel, Gnome, Blender,... and it's quite easy to find you way around.
I think that there will be neurological fatigue occurring whereby if software engineers are not actively practicing problem-solving, discernment, and translation into computer code - those skills will atrophy...
Yee, AI is not the 2x or 10x technology of the future ™ is was promised to be. It may the case that any productivity boost is happening within existing private code bases. Even still, there should be a modest uptick in noticeably improved offer deployment in the market, which does not appear to be there.
In my consulting practice I am seeing this phenomenon regularly, wereby new founders or stir crazy CTOs push the use of AI and ultimately find that they're spending more time wrangling a spastic code base than they are building shared understanding and working together.
I have recently taken on advisory roles and retainers just to reinstill engineering best practices..
I've found this to be the case with most (if not all) skills, even riding a bike. Sure, you don't forget how to ride it, but your ability to expertly articulate with the bike in a synergistic and tool-like way atrophies.
If that's the case with engineering, and I believe it to be, it should serve as a real warning.
An insidious version is AGI replacing human cognition.
To replace human thought is to replace a biological ability which progresses on evolutionary timescales - not a Moore's law approximate curve. The issue in your skull will quite literally be as useful as a cow's for solving problems... think about that.
Automating labor in the 20th century disrupts society and we've see its consequences. Replacing cognition entirely: driving, writing, decision making, and communication; yields far worse outcomes than transitioning the population from food production to knowledge work.
If not our bodies and not our minds, then what do we have? (Note: Altman's universal basic income ought to trip every dystopian alarm bell).
Whether adopted passivity or foisted actively - cognition is what makes us human. Let's not let Claude Code be the nexus for something worse.
• They don't really want to be servants.
• They have biases and preferences.
• Some of them are stupid.
• If you'd like to own an AGI that thinks for you, the AGI would also like one.
• They are people with cognition, even if we stop being.
If you think they're going to be trained on all the world's data, that's still supposing them to be an extension of AI. No, they'll have to pick up their knowledge culturally, the same way everybody else does, by watching cartoons - I mean by interactions with mentors. They might have their own culture, but only the same way that existing groups of people with a shared characteristic do, and they can't weave it out of air; it has to derive from existing culture. There's a potential for an AGI to "think faster", but I'm skeptical about what that amounts to in practice or how much use it would be to them.
Why? Does your definition postulate that people are the only thing in the universe that can measure up to us? Or the inverse, that every entity as sentient and intelligent as us must be called a person?
My opinion is that a lot of what makes us like this is physiological. Unless the developers go out of their way to simulate these things, a hypothetical AGI won't be similar to us no matter how much human-made content it ingests. And why would they do that? Why would you want to implement physical pain, or fear, or human needs, or biases and fallacies driven from our primal instincts? Would implementing all these things even be possible at the point where we find an inroad towards AGI? All of that might require creating a comprehensive human brain simulation, not just a self-learning machine.
I think it's almost certain that, while there would be some mutual understanding, an AGI would almost certainly feel like a completely different species to us.
I have sympathy with the point about physiology, though, I think being non-biological has to feel very different. You're released from a lot of the human condition, you're not driven by hormones or genes, your plans aren't hijacked to get you to reproduce or eat more or whatever animal thing, you don't have the same needs. That's all liable to alienate you from the meat-based folk. However, you're still a person.
Think of them like worker bees. Bees can solve general problems, though not on level as humans do, they are like some primitive kind of AGI. They also live and die to be servants to the queen and they don't want to be queens themselves, the reason why is interesting btw, it involves genetics and game theory.
This is highly theoretical anyways, we have no idea how to make an AGI yet, and LLMs are probably a dead end as they can't interact with the physical world.
Heads you code. Tails you review.
The only actually net positive is the Claude.md that some people maintain - it’s actually a good context dump for new engineers!
Perhaps these graphs show that management is indeed so finely tuned that they've managed to apply the AI revolution to keep productivity exactly flat while reducing expenses.
You can only utilize so many people or so much action within a business or idea.
Essentially it's throwing more stupid at a problem.
The reason there are so many layoffs is because of AI creating efficiency. The thing that people don't realize is it's not that one AI robot or GPU is going to replace one human at a one to one ratio. It's going to replace the amount of workload one person can do. Which in turn gets rid of one human employee. It's not that you job isn't taken by AI. It's started. But how much human is needed is where the new supply demand lies and how long the job lasts. There will always be more need for more creative minds. The issue is we are lacking them.
It's incredible how many software engineers I see walking around without jobs. Looking for a job making $100,000 to $200,000 a year. Meanwhile, they have no idea how much money they could save a business. Their creativity was killed by school.
They are relying on somebody to tell them what to do and when nobody's around to tell anybody what to do. They all get stuck. What you are seeing isn't a lack of capability. It's a lack of ability to control direction or create an idea worth following.
The layoffs are primarily due to over-hiring during the pandemic and even earlier during the zero-interest-rate period.
AI is used as a convenient excuse to execute layoffs without appearing in a bad position to the eyes of investors. Whether any code is actually generated by AI or not is irrelevant (and since it’s hard to tell either way, nobody will be able to prove anything and the narrative will keep being adjusted as necessary).
Nothing to do with AI.
Interest rates are still relatively high.
The reason people take jobs comes down to economics, not "creativity".
I don't, but at least it is somewhat logical. If you truly believe that, you wouldn't necessarily want to hire more developers.
99% of the draw of AI is cutting labor costs, and hiring goes against that.
That said, I don't believe AI productivity claims, just pointing out a factor that could theoretically contribute to your hypothetical.
But if your business is making software it’s hard to argue you only need a constant amount of software. I’ve certainly never worked at a software company where the to-do list was constant or shrinking!
I use Grok, Claude and Gemini every day, these "tools" are very useful to me (in the sense of how google and wikipedia changed the game) and I watch the LLM space closely, but what I'm seeing in terms of relative improvement is far removed from all the promises of the CEOs of these companies... Like, Grok 4 was supposed to be "close to AGI" but compared to Grok 3 it's just a small incremental improvement and the same goes for others...
An alternative theory is that writing code was never the bottleneck of releasing software. The exploration of what it is you're building and getting it on a platform takes time and effort.
On the other hand, yeah, it's really easy to 'hold it wrong' with AI tools. Sometimes I have a great day and think I've figured it out. And then the next day, I realize that I'm still holding it wrong in some other way.
It is philosophically interesting that it is so hard to understand what makes building software products hard. And how to make it more productive. I can build software for 20 years and still feel like I don't really know.
AI is just a convenient excuse to lay off many rounds of over-hiring while also keeping the door open for potential investors to throw more money into the incinerator since the company is now “AI-first”.
This is an insightful observation.
When working on anything, I am asked: what is the smallest "hard" problem that this is solving ? ie, in software, value is added by solving "hard" problems - not by solving easy problems. Another way to put it is: hard problems are those that are not "templated" ie, solved elsewhere and only need to be copied.
LLMs are allowing the easy problems to be solved faster. But the real bottleneck is in solving the hard problems - and hard problems could be "hard" due to technical reasons, or business reasons or customer-adoption reasons. Hard problems are where value lies particularly when everyone has access to this tool, and everyone can equally well create or copy something using it.
In my experience, LLMs have not yet made a dent in solving the hard problems because, they dont really have a theory of how something really works. On the other hand, they have really helped boost productivity for tasks that are templated .
> That’s only true when you’re in a large corporation. When you’re by yourself, when you’re the stakeholder as well as the developer, you’re not in meetings. You're telling me that people aren’t shipping anything solo anymore? That people aren’t shipping new GitHub projects that scratch a personal itch? How does software creation not involve code?
So if you’re saying “LLMs do speed up coding, but that was never the bottleneck,” then the author is saying, “it’s sometimes the bottleneck. E.g., personal projects”
Just today I built a shovelware CLI that exports iMessage archives into a standalone website export. Would have taken me weeks. I'll probably have it out as a homebrew formula in a day or two.
I'm working on an iOS app as well that's MUCH further along than it would be if I hand-rolled it, but I'm intentionally taking my time with it.
Anyway, the post's data mostly ends in March/April which is when generative AI started being useful for coding at all (and I've had Copilot enabled since Nov 2022)
That said I’ve had similar misgivings about the METR study and I’m eager for there to be more aggregate study of the productivity outcomes.
That was 5 months ago, which is 6 years in 10x time.
That's some pretty bad math.
But yes, it isn't making software get made 10x faster. Feel free to blow that straw man down (or hype influencer, same thing.)
e.g. I liked GitHub Copilot but didn't find it to be a game changer. I tried Cursor this year and started to see how AI can be today.
Im curious what the author’s data and experiment would look like a year from now.
That sure doesn't sound like 10x
"So, here’s labor productivity growth over the 25 years following each date on the horizontal axis [...] See the great productivity boom that followed the rise of the internet? Neither do I. [...] Maybe the key point is that nobody is arguing that the internet has been useless; surely, it has contributed to economic growth. The argument instead is that its benefits weren’t exceptionally large compared with those of earlier, less glamorous technologies."¹
"On the second, history suggests that large economic effects from A.I. will take longer to materialize than many people currently seem to expect [...] And even while it lasted, productivity growth during the I.T. boom was no higher than it was during the generation-long boom after World War II, which was notable in the fact that it didn’t seem to be driven by any radically new technology [...] That’s not to say that artificial intelligence won’t have huge economic impacts. But history suggests that they won’t come quickly. ChatGPT and whatever follows are probably an economic story for the 2030s, not for the next few years."²
¹ https://www.nytimes.com/2023/04/04/opinion/internet-economy....
² https://www.nytimes.com/2023/03/31/opinion/ai-chatgpt-jobs-e...
The ways AI is being used now will make this a lot worse on every front.
Background: I'm building a python package side project which allows you to encode/decode messages into LLM output.
Receipts: the tool I'm using creates a markdown that displays every prompt typed, and every solution generated, along with summaries of the code diffs. You can check it out here: https://github.com/sutt/innocuous/blob/master/docs/dev-summa...
Specific example: Actually used a leet-code style algorithms implementation of memo-ization for branching. This would have taken a couple of days to implement by hand, but it took about 20 minutes to write the spec and 20 minutes to review solutions and merge the solution generated. If you're curious you can see this diff generated here: https://github.com/sutt/innocuous/commit/cdabc98
On the other hand, I do understand that the things the LLMs are really great at is not actually all that spectacular to monetize ... and so as a result we have all these snake oil salesmen on every corner boasting about nonsensical vibecoding achievements, because that's where the real money would be ... if it were really true ... but it is not.
The reason it doesn't show up online is that I mostly write software for myself and for work, with the primary goal of making things better, not faster. More tooling, better infra, better logging, more prototyping, more experimentation, more exploration.
Here's my opensource work: https://github.com/orgs/go-go-golems/repositories . These are not just one-offs (although there's plenty of those in the vibes/ and go-go-labs/ repositories), but long-lived codebases / frameworks that are building upon each other and have gone through many many iterations.
Also, a good chunk of my personal OSS projects are AI assisted. You probably can't tell from looking at them, because I have strict style guides that suppress the "AI style", and I don't really talk about how I use AI in the READMEs. Do you also expect I mention that I used Intellisense and syntax highlighting too?
If someone builds a faster car tomorrow, I am not going to go to the office more often.
For example, I've built 5-6 iphone apps, but they're kind of one-offs and I don't know why I would put them up on the app store, since they only scratch my own itches.
Do internal, narrow purpose dev tools count as shipped code?
I linked some examples higher up, but I've been maintaining a lot of packages that I started slightly before chatgpt and then refactored and worked on as I progressively moved to the "entirely AI generated" workflow I have today.
I don't think it's an easy skill (not saying that to make myself look good, I spent an ungodly amount of time exploring programming with LLMs and still do), akin to thinking at a strategic level vs at a "code" level.
Certain design patterns also make it much easier to deal with LLM code: state reducers (redux/zustand for example), event-driven architectures, component-based design systems, building many CLI tools that the agent can invoke to iterate and correct things, as do certain "tools" like sqlite/tmux (by that I mean just telling the LLM "btw you can use tmux/sqlite", you allow it to pass hurdles that would otherwise just make it spiral into slop-ratatouille).
I also think that a language like go was a really good coincidence, because it is so amenable to LLM-ification.
But if we expect the ratio of this sort of private code to publicly-released code to remain relatively stable, which I think is a reasonable expectation, then we'd expect there to be a proportional increase in both private and public code as a result of any situation that increased coding productivity generally.
So the absence of a notable increase in the volume of public code either validates the premise that LLMs are not actually creating a general productivity boost for software development, or instead points to its productivity gains being concentrated entirely in projects that never do get released, which would raise the question of why that might be.
No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works. But, I guess that's not a priority anymore if you're willing to sacrifice $2400 a year to Anthropic.
> No, but I expect my software to have been verified for correctness, and soundness by a human being with a working mental model of how the code works.
This is not exclusive with AI tools:
- Use AI to write dev tools to help you write and verify your handwritten code. Throw the one-off dev tools in the bin when you're done.
- Handwrite your code, generate test data, review the test data like you would a junior engineer's work.
- Handwrite tests, AI generate an implementation, have the agent run tests in a loop to refine itself. Works great for code that follows a strict spec. Again, review the code like you would a junior engineer's work.
I generate between 10-100k lines of code per day these days. But is that a measure of productivity? Not really...
That’s absolute nonsense.
"I'm a person who hates art now...I never want to see art again. All I want to see is like, AI stuff. That's how bad it's gotten. Handmade? nuh-uh. Handmade code? ... anything by humans, just over. I'm just gonna watch pixels."
https://www.youtube.com/live/APkR4qRg1vM?si=XLGmH9uEjG08q-6x...
I watched a little more but was, uh, not impressed.
But if llms show us one thing, it’s how bad our code review tools are. I have a set of tree sitter helpers that allow me to examine different parts of a PR more easily (one that allows me to diff semantic parts of the code, instead of “files” and “lines”, one that gives me stats on what subsystems are touched and crosscorrelation of different subsystems, one for attaching metadata and which documents are related to a commit, one for managing our design documents, llm-coding intermediary documents, long lasting documents, etc… the proper version of these are for work but here’s the initial yolo from Manus: https://github.com/go-go-golems/vibes/tree/main/2025-08-22/p... https://github.com/go-go-golems/vibes/tree/main/2025-08-22/c... https://github.com/go-go-golems/vibes/tree/main/2025-08-15/d... https://github.com/go-go-golems/vibes/tree/main/2025-07-29/p...).
I very often put some random idea into the llm slot machine that is manus, and use the result as a starting point to remold it into a proper tool, and extracting the relevant pieces as reusable packages. I’ve got a pretty wide treesitter/lsp/git based set of packages to manage llm output and assist with better code reviews.
Also, every llm PR comes with _extensive_ documentation / design documents / changelogs, by the nature of how these things work, which helps both humans and llm-asssisted code review tools.
“ Write a streaming go yaml parsers based on the tokenizer (probably use goccy yaml if there is no tokenizer in the standard yaml parser), and provide an event callback to the parser which can then be used to stream and print to the output.
Make a series of test files and verify they are streamed properly.”
This is the slot machine. It might work, it might be 50% jank, it might be entire jank. It’ll be a few thousand lines of code that I will skim and run. In the best case, it’s a great foundation to more properly work on. In the worst case it was an interesting experiment and I will learn something about either prompting Manus, or streaming parsing, or both.
I certainly won’t dedicate my full code review attention to what was generated. Think of it more as a hyper specific google search returning stackoverflow posts that go into excruciating detail.
https://chatgpt.com/share/68b98724-a8cc-8012-9bee-b9c4a77fe9...
If I’m working against a deadline I feel more comfortable spending time on research and design knowing I can spend less time on implementation. In the end, it took the same amount of time, though hopefully with an increase of reliability, observability, and extendibility. None of these things show up in the author’s faulty dataset and experiment.
https://github.com/go-go-golems/ai-in-action-app/blob/main/c...
There are many reasons for your experience, and I am glad you are having them! That's great!
But the fact remains, overall we aren't seeing an exponential or even step function in how much software is being delivered!
At this point, one is gaining with each model release or they are not.
Lets see in 2035 who was right and who was wrong. My bet is the people who are not gaining right now are not going to like the situation in 2035.
Where i expect to see a lot of those metrics of feeling fast come from, is from people who may have less coding experience, and with AI are coding way above their level.
My brother in law asks for a nice product website, i just feed his business plan into a LLM, do some fine tuning on the results, and have a good looking website in a hour time. If i did it myself manually, just take me behind a barn as those jobs are so boring and take for ages. But i know that website design is a weakness of mine.
That is the power of LLMs. Turn out quick code, maybe offer some suggestion you did not think about, but ... it also eats time! Making your prompts so that the LLM understands, waiting for the result, ... waiting ... ok, now check the result, can you use it? O no, it did X, Y, Z wrong. Prompt again ... and again. And this is where your productivity goes to die.
So when you compare a pool of developer feedback, your going to get a broad "it helps a lot", "some", "is worse then my code", ... mix in with the prompting, result delays etc...
It gets even worse with Agent / Vibe coding, as you just tend to be waiting, 5, 10min for changed to be done. You need to review them, test them, ... o no, the LLM screwed something up again. O no, it removed 50% of my code. Hey, where did my comments go. And we are back to a loss of time.
LLMs are a tool... But after a lot of working with them, my opinion is to use them when needed but do not depend on them for everything. I sometimes look with cow eyes when people say they are coding so much with LLMs and spending 200, or more bucks per month.
They can be powerful tools, but i feel that some folks become so over dependent on them. And worst is my feeling that our juniors are going to be in a world of hurt, if their skills are more LLM monkey coding (or vibe coding), then actually understanding how to code (and the knowledge behind the actual programming languages and systems).
When I tried to code again, I found I didn't really have the patience for it -- having to learn new frameworks, APIs, languages, tricky little details, I used to find it engrossing: it had become annoying.
But with tools like Claude Code and my knowledge about how software should be designed and how things should work, I am able to develop big systems again.
I'm not 20% more productive than I was. I'm not 10x more productive than I was either. I'm infinity times more productive because I wouldn't be doing it at all otherwise, realistically: I'd either hire someone to do it, or not do it, if it wasn't important enough to go through the trouble to hire someone.
Sure, if you are a great developer and spend all day coding and love it, these tools may just be a hindrance. But if you otherwise wouldn't do it at all they are the opposite of that.
Others are for start-ups that are pre-money, pre-revenue where I can build things myself without having to deal with hiring people.
In a larger organization, certainly I'd delegate to other people, but if it's just for me or new unfunded start-ups, this is working out very well.
And it's not that I "can no longer program". I could program, it's just that I don't find the nuts and bolts of it as interesting as I used to and am more focused on functionality, algorithm, and UI.
Let's take the following scenario for the sake of argument: a codebase with well-defined AGENTS.md, referencing good architecture, roadmap, and product documentation, and with good test coverage, much of which was written by an LLM and lightly reviewed and edited by a human. Let's say for the sake of argument that the human is not enjoying 10x productivity despite all this scaffolding.
Is it still worthwhile to use LLM tooling? You know what, I think a lot of companies would say yes. There are way too many companies whose codebases lack testing and documentation, that are too difficult to on-board new engineers and have too high risk if the original engineers are lost. The simple fact that LLMs, to be effective, force the adaptation of proper testing and documentation is a huge win for corporate software.
Yes, but the point of this article is surely that on average if it's working, there would be obvious signs of it working by now.
Even if there are statistical outliers (ie. 10x productivity using the tools), if on average, it does nothing to the productivity of developers, something isn't working as promised.
I believe it's a productivity boost, but only to a small part of my job. The boost would be larger if only had to build proof-of-concepts or hobby projects that don't need to be reliable in prod, and don't require feedback and requirements from many other people.
Just spit balling here, but it sure feels similar.
I have the same experience as OP, I use AI every day including coding agents, I like it, it's useful. But it's not transformative to my core work.
I think this comes down to the type of work you're doing. I think the issue is that most software engineering isn't in fields amenable to shovelware.
Most of us either work in areas where the coding is intensely brownfield. AI is great but not doubling anyone's productivity. Or, in areas where the productivity bottlenecks are nowhere near the code.
If AI were really making people 10x more productive, given the number of people who want to make games, you’d expect to see more than a few percent increase year over year.
That said, I’m skeptical that AI is as helpful for commercial software. It’s been great for in automating my workflow because I suck at shell scripting and AI is great at it. But most of the code I write I honestly halfway don’t know what I’m going to write until I write it. The prompt itself is where my thinking goes - so the time savings would be fairly small, but I also think I’m fairly skilled (except at scripting).
The same people who are willing to go through all the steps to release an application online are also willing to go through the extra effort of writing their own code. The code is actually the easy part compared to the rest of it... always has been.
As an analogy, can you imagine being a startup that hired a developer, and months later finding out the bulk of the new Web app they "coded" for you was actually copy&pasted open source code, loosely obfuscated, which they were passing it off as something they developed, and to which the company had IP rights?
You'd immediately convene the cofounders and a lawyer, about how to make this have never happened.
First you realize that you need to hand the lawyer the evidence (against the employee), and otherwise remove all traces of that code and activity from the company.
Simultaneously, you need to get real developers started rushing to rewrite everything without obvious IP taint.
Then one of you will delicately ask whether firing and legal action against the employee is sufficient, or whether the employee needs to sleep with the fishes to keep them quiet.
The lawyer will say this kind of situation isn't within the scope of their practice, but here's the number of a person they refer to only as 'the specialist'.
Soon, not only are you losing the startup, and the LLC is being pierced to go after your personal assets, but you're also personally going to prison. Because you were also too cheap to pay the professional fee for 'the specialist', and you asked ChatGPT to make the employee have a freak industrial shredder accident.
All this because you tried to cheap out, and spend $20 or $200 on getting some kind of code to appear in your repo, while pretending you didn't know where it came from.
Oh, and then you have the actual tech giants offering legal commitment to protect you against any copyright claims:
https://blogs.microsoft.com/on-the-issues/2023/09/07/copilot...
https://cloud.google.com/blog/products/ai-machine-learning/p...
You might be right, but the point needs to be made.
https://techcrunch.com/2024/09/30/y-combinator-is-being-crit...
Maybe investors will care, but for now they stand to make more money from "AI" gold rush startups, and don't want to be a wet blanket on "AI" at all by bringing up concerns.
From my experience, it's much easier to get an LLM to generate code for a React/Tailwind CSS web app than a mobile app, and that's why we're seeing so many of these apps showing up in the SaaS space.
In fact it looks like there were less products launched last month on PH than the same period a year ago.
https://hunted.space/monthly-overview
It's a bit hard as they're not summing by months but it looks like less to me quickly scanning it.
And as Claude Code has only really been out 3/4 months you'd be expecting launches to be shooting up week-by-week right about now as all the vibe products get finished.
They're not, see the 8 week graph:
Well... no significant effects show except for a few projects. It was really hard torturing the data to come to my manager's desired conclusion.
Its sometimes helpful when writing an email but otherwise has not touched any of my productive work.
Changing domain to writing and images and video you can see LinkedIn is awash with everyone generating everything by LLMs. The posting cadence has quickened too as people shout louder to raise their AI assisted voice over other people’s.
We’ve all seen and heard the AI images and video tsunami
So why not software (yet but soon)??
Firstly, Software often has a function and AI tool creations cannot make that work. Lovable/Bolt etc are too flakey to live up to their text to app promises. A shedload of horror debugging or a lottery win of luck is required to fashion an app out of that. This will improve over time but the big question is, by enough?
And secondly, like on LinkedIn above: perhaps the standards of the users will drop? LinkedIn readers now tolerate the llm posts, it is not a mark of shame. Will the same reduction in standards in software users open the door to good-enough shovelware?
How much of it is to be blamed on AI, and how much on a culture of making users test their products, I do not know.
Everything has been enshittified so much that nothing phases them anymore.
It'll increase incremental developments manyfold. A non-programmer spending a few hours on AI to make their workflow better and easier and faster. This is what everyone here keeps missing. It's not the programmers that should be using AI; it's 'regular' people.
If AI enables regular folks to make programs, even if the worst quality shovelware, there should’ve been an explosion in quantity. All the programs that people couldn’t made, they would start making them in the past two years.
I'm not sure how that challenges the point of the article, which is that metrics of the total volume of code being publicly released is not increasing. If LLMs are opening the door to software development for many people whose existing skills aren't yet sufficient to publish working code, then we'd expect to see a vast expansion in the code output by such people. If that's not happening, why not?
On my computer. Once I've built something I often realize the problems with the idea and abandon the project, so I'm never shipping it.
In fact, being able to throw it out like this is a big time saver in itself. I've always built a lot of projects but when you've spent weeks or months on something you get invested in it, so you ship it even though you no longer believe in it. Now when it only takes a couple of days to build something you don't have the same emotional attachment to it.
My question to HN is... can LLMs do this? Can they convert all the unsafe c-string invocations to safe. Can they replace system calls with posix calls. Can they wrap everything in a smart pointer and make sure that mutex locks are added where needed.
There’s a lot more code being written now that’s not counted in these statistics. A friend of mine vibe coded a writing tool for himself entirely using Gemini canvas.
I regularly vibe code little analyses or scripts in ChatGPT which would have required writing code earlier.
None of these are counted in these statistics.
And yes AI isn’t quite good enough to super charge app creation end to end. Claude has only been good for a few months. That’s hardly enough time for adoption !
This would be like analysing the impact of languages like Perl or Python on software 3 months after their release.
And if it's not that, then the silly money valuations don't make any sense.
I'd also be curious how the numbers look for AI generated videos/images, because social media and youtube seem absolutely flooded with the stuff. Maybe it's because the output doesn't have to "function" like code does?
Grammatical nit: The phrase is "neck and neck", like where two race horses are very close in progress
But my workflow is anything but "let her rip". It's very calculated, orderly, just like mastering any other tool. I'm always in the loop. I can't imagine someone without serious experience getting good stuff, and when things go bad, oh boy you're bringing a firehose of crap into your org.
I have a junior programmer who's a bright kid but lacking a lot of depth. Got him a Cursor subscription, tracking his code closely via PRs and calling out the BS but we're getting serious work done.
I just can't see how this new situation calls for less programmers. It will just bring about more software, surely more capable software after everyone adjusts.
But according to the graphs in the article, after three years of LLM chatbots and coding assistants, we're seeing exactly the same rate of new software...
So who pays for AIs for developers? Mostly corpos. And the speed of individual developer was never a limiting factor in corpos. Average corporate development was always 10 times slower than indie. So even doubling it won't make any impression.
I don't know if I'm faster with AI at a specific task, but I know that I'm doing things I wouldn't touch because I hate the tedium. And I'm doing them while cooking and eating dinner and thinking about wider context and next things to come. So for me it feels worth it.
I think it might be something like with cars and safety. Any car safety improvements are going to be offset by the drivers driving faster and more recklessly. So maybe any speed improvements that AI might make for the project is nullified by developers doing things they would just skip without it.
> The impact on human lives is incredible. People are being fired because they’re not adopting these tools fast enough. People are sitting in jobs they don’t like because they’re afraid if they go somewhere else it’ll be worse. People are spending all this time trying to get good at prompting and feeling bad because they’re failing.
The shovelware was always there. And it always will be. But that's doesn't mean it's splurting out faster, because that's not what AI does. Hell, if anything I expect that there's less visible shovelware because when it does get created, it's less obvious (and perhaps higher quality).
At some point, the quality of uninspired projects will be lifted up by the baseline of quality that mainstream AI allows. At what point is that "high enough that we can't tell what's garbage"? We've perhaps found ourselves at or around that point.
For example, in Steam, it costs $100 to release a game. You may extend your game with what's called a DLC and that costs $0 to release. If I were to build shovelware with especially with AI-generated content, I'd more keen to make a single game with a bunch of DLC.
For game development, integration of AI into engines is another barrier. There aren't that many choices of engines that gives AI an interface to work with. The obvious interface is games that can be entirely build with code (e.g., pygame; even Godot is a big stretch)
Where I have found them very useful are for one-off scripts and stuff I need done quick and dirty, that isn't too complex and easily verifiable (so I can catch the mistakes it makes, and it does make them!), and especially in languages I don't know that well or don't like (i.e., bash, powershell, javascript)
No need to learn a programming language, wow, anyone can be a programmer now. A few projects come out of it, people marvel at how efficient it was, and it fizzles out and programmers continue writing code.
If anything, things like visual programming did more than AI does now. For games, if you want to see the shovelware, look at Flash, RPG maker, etc... not AI. On the business side of things, Excel is king. Can you get your vibe coded app out faster than by using Flash or Excel?
Instead I’m not waiting for something like Linux on smartphones to come so soon.
I guess someone could try a prompt of "generate a patch set from Linux tree X to apply to mainline Linux for this CPU".
I see pseudo-scientific claims from both sides of this debate but this is a bit too far for me personally. "We all know" sounds like Eternal September [1] kind of reasoning. I've been in the industry about as long as the article author and I think he might be looking with rose-tinted glasses on the past. Every aging generation looks down at the new cohort as if they didn't go through the same growing pains.
But in defense of this polemic, and laying out my cards as an AI maximalist and massive proponent of AI coding, I've been wondering the same. I see articles all the time about people writing this and that software using these new tools and it so often is the case they never actually share what they built. I mean, I can understand if someone is heads-down cranking out amazing software using 10 Claude Code instances and raking in that cash. But not even to see one open source project that embraces this and demonstrates it is a bit suspicious.
I mean, where is: "I rewrote Redis from scratch using Claude Code and here is the repo"?
This is one of my big datapoints in the skepticism, there's all these articles about how individual developers are doing amazing things, but almost no data points about the increase of productivity as a result.
Meanwhile I see WhatsApp sunsetting their native clients and making everything a single web-based client. I guess they must not be using LLMs to code if they can’t cope with maintaining the existing codebases, right?
There's really a lot to get from this "tool". Because in the end its a tool, and knowing how to use it is the most important aspect of it. It takes time, iteration, and practice to understand how to effectively use it
When you consider this, "generate me a whole repo" is trivially garbage and not meeting the measurement metric. However having AI autocomplete "getUser(..." clearly IS productive.
Now is that a 0.1% increase, 1%, or 10%? That I can't tell you.
Even if AI made them more productive, it's on a person to decide what to build and how to ship, so the number (and desire) of humans is a bottleneck. Maybe at some point AI will start buying up domains and spinning up hundreds of random indiehacker micro-SaaS, but we're not there. Yet.
This isn't likely to happen -- if the problem is very specific, you won't be able to sufficiently express it in natural language. We invented programming languages precisely because natural languages are completely unsuited for the task of precisely specifying a problem.
So how are you going to express the specificity of the problem to the LLM in natural language? Try, and you'll discover their shortcomings for yourself. Then you'll reinvent programming languages.
What if someone came out with a study saying they had a tool to make the fastest doctors and lawyers? You'd say that doesn't even make sense, what kinds of doctors doing what kinds of work?
AI coding isn't some across the board, helps everyone do anything kind of tool.
Maybe sometime soon we'll stop strawmanning this.
The arrival of the Trough is predicated by the amount of lies and utter bullshit that have been shoveled out during the earlier parts of the cycle. So while it's unfortunate that the real goods don't get delivered for years and years after they might have been, it's typically and often entirely the fault of the people on the train that this has happened.
There's an awful lot of utter bullshit in the AI hype.
1. Only a handful of devs use LLMs
2. For every developer getting less productive with LLMs, there must be developers that get more productive to keep the trend
I have prompt docs precisely on SOLID, TDD and all kinds of design patterns… but yes I see a lot of untested code these days.
AI has been incredibly helpful at analyzing existing, unknown to me, projects; basically for debugging and searching in these repo’s.
Archived here: https://archive.is/WN3iu
Was the author making games and other apps in 30 hours? Because that seems like a 4 month project?
* METR was at best a flawed study. Repo-familiarity and tool-unfamiliarity being the biggest points of critique, but far from the only one
* they assume that all code gets shipped as a product. Meanwhile, AI code has (at least in my field of view) led to a proliferation of useful-but-never-shipped one-off tools. Random dashboards to visualize complex queries, scripts to drive refactors, or just sheer joy like "I want to generate an SVG of my vacation trip and consume 15 data sources and give it a certain look".
* Their own self-experiment is not exactly statistically sound :)
That does leave the fact that we aren't seeing AI shovelware. I'm still convinced that's because commercially viable software is beyond the AI complexity horizon, not because AI isn't an extremely useful tool
They didn't claim it was flawless, they had just brought it up because it caused them to question their internal narrative of their own productivity.
> * Their own self-experiment is not exactly statistically sound :)
They didn't claim it was.
> * they assume that all code gets shipped as a product.
The author did not assume this. They assumed that if AI is making developers more productive, that should apply to shovelware developers. That we don't see an increase in shovelware post-AI, makes it very unlikely AI brings an increase in productivity for more complex software.
Somehow, in cyber, everyone believes that transformers will generate better answers than not to use the 10 most common passwords. It's like the whole knowledge about decision making theory, neural nets, GANs, LSTMs etc completely got wiped out and forgotten within less than 10 years.
I understand the awesomeness of LLMs while debugging and forensics (they are a really good rubberduck!), but apart from that they're pretty much useless because after two prompts they will keep forgetting if/elseif/else conditions, and to check those boundaries is the mission of the unlucky person that has to merge that slopcode later.
I don't understand how we got from TDD and test case based engineering to this bullshit. It's like everyone in power was the wrong person to be in that position in the first place, and that statistically no lead engineers ever will be a C-staff or SVP or whatever corporate manager level.
While the AI bubble is bursting, I will continue to develop with TDD practices to test my code. Which, in return, has the benefit of being able to use LLMs to create nice templates as a reasonable starting point.
My personal hypothesis is that when using LLMs, you are only faster if you would be doing things like boilerplate code. For the rest, LLMs don't really make you faster but can make your code quality higher, which means better implementation and caching bugs earlier. I am a big fan of giving the diff of a commit to an LLM that has a file MCP so he can search for files in the repo and having it point any mistakes I have made.
Most users have 40-80 apps installed and use 9 a day, 20 a month(1). The shitty iOS subscription trend killed off the hobby of 'app collecting'.
Have I created large commercial Ai-coded projects? No. Did I create 80+ useful tools in hours/days that I wouldn't have otherwise? Hellz yeah!
Would I publish any of these on public github? Nope! I don't have the time nor the inclination to maintain them. There's just too many.
My shovelware "Apps" reside on my machine/our intranet or V0/lovable/bolt. Roughly ~25% are in active daily use on my machine or in our company. All tools and "apps" are saving us many hours each week.
I'm also rediscovering the joy of coding something useful, without writing a PRD for some intern. Speaking of which. We no longer have an intern.
- Built a windows desktop app that scans local folders for videos and automatically transcribes the audio, summarises the content into a structured JSON format based on screenshots and subtitles, and automatically categorises each video. I used it on my PC to scan a couple of TB of videos. Has a relatively nice interface for browsing videos and searching and stores everything locally in SQLite. Did this in C# & Avalonia - which I've never used before. AI wrote about 75% of the code (about 28k LOC now).
- Built a custom throw-away migration tool to export a customers data from one CRM to import into another. Windows app with basic interface.
- Developed an AI process for updating a webform system that uses XML to update the form structure. This one felt like magic and I initially didn't think it would work, but it only took a minute to try. Some background - years ago I built a custom webform/checklist app for a customer. They update the forms very rarely so we never built an interface for making updates but we did write 2 stored procs to update forms - one outputs the current form as XML and another takes the same XML and runs updates across multiple tables to create a new version of the form. For changes, the customer sends me a spreadsheet with all the current form questions in one column and their changes in another. It's normally just wording changes so I go through and manually update the XML and import it, but this time they had a lot of changes - removing questions, adding new ones, combining others. They had a column with the label changes and another with a description of what they wanted (i.e. "New Question", "Update label", "Combine this with q1, q2 and q3", "remove this question"). The form has about 100 questions and the XML file is about 2500 lines long and defines each form field, section layout, conditional logic, grid display, task creation based on incorrect answers etc, so it's time consuming to make a lot of little changes like this. With no expectation of it working, I took a screenshot of the spreadsheet and the exported XML file and prompted the LLM to modify the XML based on the instructions in the spreadsheet and some basic guidelines. It did it close to perfect, even fixing the spelling mistakes the customer had missed while writing their new questions.
- Along with using it on a daily basis across multiple projects.
I've seen the stat that says developers "...thought AI was making them 20% faster, but it was actually making them 19% slower". Maybe I'm hoodwinking myself somehow, but it's been transformative for me in multiple ways.
Or, alternatively, exposure to our robot overlords makes you less discerning, less concerned with, ah, whether the thing is correct or not.
(This _definitely_ seems to be a thing with LLM text generation, with many people seemingly not even reading the output before they post it, and I assume it's at least somewhat a thing for software as well.)
I didn't know Mike Judge was such a polymath!
For experienced engineers, I'm seeing (internally in our company at least) a huge amount of caution and hesitancy to go all-in with AI. No one wants to end up maintaining huge codebases of slop code. I think that will shift over time. There are use cases where having quick low-quality code is fine. We need a new intuition about when to insist on handcrafted code, and when to just vibecode.
For non-experienced engineers, they currently hit a lot of complexity limits with getting a finished product to actually work, unless they're building something extremely simple. That will also shift - the range of what you can vibecode is increasing every year. Last year there was basically nothing that you could vibecode successfully, this year you can vibecode TODO apps and stuff like that. I definitely think that the App Store will be flooded in the coming future. It's just early.
Personally I have a side project where I'm using Claude & Codex and I definitely feel a measurable difference, it's about a 3x to 5x productivity boost IMO.
The summary.. Just because we don't see it yet, doesn't mean it's not coming.
There are very simple apps I try to vibe code that AI cannot handle. It seems very good at certain domains, and others it seems complete shit at.
For example, I hand wrote a simulation in C in just 900 LOC. I wrote a spec for it and tried to vibe code it in other languages because I wanted to compare different languages/concurrency strategies. Every LLM I've tried fails, and manages to write 2x+ more code in comparatively succinct languages such as Clojure.
I can totally see why people writing small utilities or simple apps in certain domains think its a miracle. But when it comes to things like e.g. games it seems like a complete flop.
- breaking through the analysis paralysis by creating the skeleton of a feature that I then rework (UI work is a good example)
- aggressive dev tooling for productivity on early stage projects, where the CI/CD pipeline is lacking and/or tools are clumsy. (Related XKCD: https://xkcd.com/1205/)
Otherwise, I find most of my time is understanding the client requirements and making sure they don't want conflicting features – both of which are difficult to speedup with AI. Coding is actually the easy part and even if it was sped up 100x a consistent end-to-end improvement of 2x would be a big win (see Amdahl's law).
How is this "deadly" serious? It's about software developers losing well-paid, comfortable jobs. It's even less serious if AI doesn't actually improve productivity, because they'll find new jobs in the future.
Pretty much the only future where AI will turn out "deadly serious" is if it shows human-level performance for most if not all desk jobs.
This is what I always look for. Haven’t found one salient success story with a claim for success.
We might be doing just that now.
The best way to increase your ROI is to fire all your employees. How do we know we're not in the mid-release-cycle of that right now?
I'd guess game levels and assets are becoming ai slop as we speak.
>” Now, I’ve spent a lot of money and weeks putting the data for this article together, processing tens of terabytes of data in some cases. So I hope you appreciate how utterly uninspiring and flat these charts are across every major sector of software development.”
A modern animated Disney 3D animated film consistently costs over 100-200 million dollars while movies like Klaus were made for about 40 million. Japan still animates on PAPER.
At the end of the days new tools have their usecases but I think especially in creative domains (which software definitely is) old techniques aren't invalidated by the creation of new ones.
ZBrush still crushes all other sculpting apps with some very well written low level code and assembly. It doesn't even use the GPU for crying out loud. If you proposed that as your solution for a graphically intensive 3D app you'd be laughed at, but software based raseterization/simple ray tracing takes the cake here. It could handle 20 million polygons at buttery smooth framerates in 2007, and isn't burdened by the VRAM drought we're in.
Don't let people tell you new tools make the old useless.
I've been a "10xer" for 25 years. I've considered coding agents bullshit since my first encounter with Copilot. I work by having a clear mental map of every piece of my code and knowing exactly how everything works, to the smallest detail, and how it interacts with every other part.
Anyway, all that and a nickel. Yesterday I fired up Claude Code for the first time. I didn't ask it to build me a game or anything at a high level. Nor to evaluate an existing code base. No... I spent about 2 hours guiding it to create a front-end SPA framework that does what my own in-house SPA framework does on the front end, just to see how it would perform at that. I approved every request manually and interrupted every time I spotted a potential issue (which were many). I guided it on what functions to write and how they should affect the overall navigation flow, rendering flow, loading and error-handling.
In other words, I knew what I wanted to write to a T, because it's code I wrote in 2006 and have refactored and ported many times since then... about 370 commits worth to this basic artifact, give or take.
And it pretty much got there.
Would I have been able to prompt it to write a system like that if I hadn't written the system myself over and over again? Probably not. But it did discern the logical setup I was going for (which is not at all similar to what you're thinking if you're coming from React or another framework like that), and it wrote code that is almost identical in its control structures to what I wrote, without me having to do much besides tell it in plain English what should control what, how, when and in what order.
I'm still not convinced it would save me time on something totally new, that I didn't already know the final shape of.
But I do suspect that a reason all this "vibe coding" hasn't led to an explosion of vaporware is that "vibe coding" isn't being done by experienced coders. I suspect that if you're letting changes "flash across the screen" without reading them, that's most of the difference between a failed prompt and one that achieves the desired result.
Like, I saw it do things like create a factory class that took a string name and found the proper component to load. I said, "refactor that whole thing to a component definition interface with the name in it, make a static object of those and use that to determine what screen to load and all of its parameters." And it did, and it looked almost the same as what I wrote back in the day.
Idk. I would not want my job to become prompting an LLM. I like cracking my knuckles and writing code. But I think the mileage you get may be dictated by whether you are trying to use it as a general-purpose "make this for me" engine, for shovelware, in which case it will fail hard, versus whether you are using it as a stenographer translating a sentence of instructions into a block of control flow.