Eight Myths on Software Engineering and GenAI
queue.acm.org
queue.acm.org
This is such a weird point to make that doesn't become correct just because everyone makes it, all the time. Why clean the ocean if some magic future tech will clean them? Why save the world now if some benevolent AI is 'just around the corner' and will do it for us? And people have been making this point for years now, and it's not like my job got any easier. I just got more AI.
https://www.poetryfoundation.org/poems/51294/waiting-for-the...
And I say that as someone who uses Claude Code in complex environments almost hourly; I, as the human, still have to do the thinking as Claude still 'can't jump' [1] and I have seen no evidence that they (or similar AI, any time soon) will 'jump' like a human brain does.
I still have to do quite a bit of thinking but the amount of of thinking I do per task is trending down. I agree LLMs are not good at abduction but very few humans are either and very few jobs/tasks require it. I can't talk for researchers jobs though. But perhaps fewer researchers would be desired by these labs (not none).
I, too, am glad that few humans seem good at abduction.
Don't worry, I'm sure you'll hit your goal of zero thinking soon!
The IQ willl drop if we just become mechanical acritical people the same way muscles get worse if you do not exercise.
I say this as someone who's watched a bunch of woodworking videos but hasn't actually done this myself :)
Self-service and office suite software is largely responsible for this.
Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career. Where do you see the opportunity where I just see a bleak one where anyone can do this stuff by typing or talking to AI? Myself, after 17 years in the field I am begrudingly back in school for a new medical career. As well, anytime an IT recruiter reaches out I am getting responses back only after under-cutting the hourly rate I use to demand and what others probably are still trying to get. And with it feels even bleaker as it becomes a race to the bottom!
The problem, for the profession, is that the set of people who can really do this stuff by typing is close to "all of them". I'm not seeing anyone struggle with using AI. I see struggles from professional software developers because they are trying to get quality output, but if you don't have a bar for quality, just about everyone can create their own software.
A poster a few months ago had a Show HN about his 7 year old kid, barely able to read, who was happily vibing up games.
> Overall, I'd like to understand those who have a positive outlook on design and software engineering as a career.
I think until the market better achieves some equilibrium, there is no way general software programming (sorry “engineering”) should be considered as a career. That said, there will always be opportunities in particular markets or specialties.
Btw, I think the discussion of Einstein's career in the paper you link is historically wrong in many respects, particularly the argument about 'weak signal'. Einstein was in fact working on some of the most mainstream and widely discussed problems in physics of the day, he is admired for the creativity of his solutions to those problems, and much of his work built incrementally on ideas and breakthroughs that came (long) before (as all research does).
Article suggests that a central motivation of Einstein's work was resolving action-at-a-distance in Newtonian mechanics - yet Maxwell introduced the same Lagrangian field theories for electromagnetism we use today 50 years earlier to solve the same problem for Farraday's laws of electromagnetism. Similar wave equations existed even earlier. Heaviside in 1893 extended this technique to gravity (matching 'weak field' GR) 20 years earlier. So this is perhaps the one aspect of gravity that had actually already been solved before Einstein. Authors might be conflating his work on action-at-a-distance in QM.
Einstein's GR extended the linear 'weak field' understanding of gravity to include the non-linear self-referential case where masses themselves create gravity. This was mathematically incredibly difficult but was necessary precisely because SR's mass energy equivalence created so many strong signals that were unresolved. For example: if finite energy is mass, then mass changes as objects accelerate past a large mass like a start, and hence their propagation in space could not be explained by linear EM style field equations. Many such considerations were causing very 'strong signals' in SR, and there were analogous problems in QM atomic models being developed at the same time.
SR was also a solution to a problem that was actively being worked by many of the leading physicists of the day. SR actually does match Newtonian mechanics for a single observer - it resolves contradictions in the case of separate observers, by allowing them to assign different values to the speeds, masses, etc of objects such that each object appears to follow Newtonian mechanics for each observer. Again, this was necessary because of a lot of contradictions related to the behavior of light that had been well-known for ~20 years at the time.
Personally, I don't consider this kind of reasoning to be beyond the capabilities of future LLMs (even current LLMs if the task was broken into technical rather than philosophical problems). Personally, I doubt that such problems could stand open for 20+ years waiting for a creative genius to solve them in the modern world.
And don't get me started on the philosophy.
What kind of answers were they expecting to get?
> > Mathematician Richard Hamming used to ask scientists in other fields "What are the most important problems in your field?" partly so he could troll them by asking "Why aren't you working on them?" and partly because getting asked this question is really useful for focusing people's attention on what matters.
> I imagine someone being asked this question, and how they should respond. I think like so - ‘Fuck off Richard’.
> This is partly because I imagine this question being asked in a kind of snarky, gotcha kind of way, with some sort of nerdy superiority. Like ‘ha your behaviour is inconsistent with your implied preferences, you idiot, do you even von Neumann–Morgenstern?’
Generating a good looking UI for example, is so much easier now with LLM.
For a joke I asked ChatGPT yesterday to make a short promoimage for a 'joke' idea i had, it was above avg. I have for sure seen worse Marketing Images than what ChatGPT generated.
It looked similiar to plenty of other Marketing Images but its not that anyone cares.
And those same architects are quietly extracting real productivity from GenAI.
And even this write up skips that info by waving, “Some people…”
Which kind of answers the original question "why bother working?" with "because now, I can do a bit more than before".
I also see bad quality (in code, documents, presentations). It comes from people that had no clue how to do something before and now they imagine that just asking Claude is solving well the problem. And is annoying (and hard) to explain to it them, and then they get frustrated.
They learn the concept of things and how to do them because this is better compression than learning concepts one by one.
Which means, if an LLM 'learns' the concept of a poem, it can put everything into the formad of a poem instead of learning a billion poems.
When anthropic looked at how an LLM does addition it found it had some mental math heuristics that might or might not always work. The LLM hadn't learned the concept of addition. It had learned some heuristics that might work for some numbers. The result is that LLM's cannot add numbers reliably because they have not learned the concept of addition.
Might be an architecture issue or a parameter size issue that it didn't learn to do math like a caculator.
But look at your own math skills: How many numbers / how big of numbers can you keep in your head? How far is this heuristic away from how much a human learned until you start using pen and paper or a caculator?
I think it is a great point to make, because if everyone really believed that AIs will do everything without human intervention in a handful of years, as the marketing repeats again and again (AGI, singularity, etc.) and have been saying for years... why then get bothered?
Because we DO know LLMs have their hallucinations, limitations, perform tasks not previously seen way worse than humans, etc. And it seems that, for now, there is not a good or magic solution to it, it is inherent limitations of the paradigm.
Yes, you can feed more and more and more (curated data) and eventually make AIs excellent at task X or Y, but then you spend your time specializing those engines. So the work does not really disappear, it just shifts and you make it more replicable for a bound set of problems.
Needless to say that at some point I prefer to learn (and combine with AIs, it is ok) than acritically getting inputs from something until I become totally useless.
Unless we have a paradigm for which a fully autonomous AI can do everything, this will just become improving our productivity in some ways, with all the in-between bottlenecks that it has.
Bc they are aware of the marketing and limitations. If they did believe it, then they would switch area of research.
This is such a weird point to make. We are currently ( only ) discovering that paradigm; we are not inventing anything. We found a bunch of laws that produce rather cool results but our paradigm is incomplete which leads more or less wordy or frame-rich weird stuff like hallucinations, singularity and so on ... it's childish, really and on that funny pseudo-profound, pseudo-intellectual, pseudo-spiritual ( personal opinion, if it gets you horny, you go, baby ) "universe consciousness unity, Rick James, bitch" level ...
Our bodies and minds need proper AI, not all the stuff we already outsource to middle and/or passionate men and women. Other species on the planet would certainly like to see us get augmented by AI so we can solve as many survivability issues as possible to keep as many ecosystems running long enough ... whatever that means but whether animals and plants are aware of chance and potential is another philosophical debate.
To individuals, software is a hammer and chisel, a knife, a brush and canvas, pen and paper, a reading help, and to a good amount of people it's a microscope and a fine scalpel.
To collectives, it's a tool to work on consensus and conventions, to share and gather.
It's baby steps for civilizations and it looks like our particular species is gonna get stuck in a puddle of our own monkey shit, with bottles of champagne in our hands and monkeys grinding up and down the few ivory towers in proximity.
> why then get bothered
Humans are on different levels. Most have decided that "nature realized the/a bug and wanted someone dead" or "their survival is a matter of chance" is not acceptable at all and some people decided that sabotage, poison, abuse, rape, murder are acceptable means to get chicken shit ...
The "paradigm" of life is far from explored/discovered, so we simply can't content ourselves with presumptions about inherent limitations of the LLM and AI paradigm for any other reason than to uncover ( not invent ) other parts of the paradigm.
We are happy with what AI can do for us but "AIs will do everything without human intervention" sounds weird because babies are born and the older they get and the less sabotaged ( vs influence, cultural manipulation ) they get to grow up, the more breadth and depth humans want to experience. For this they need to learn and use their hands & fingers. They need to feed body and mind to find what triggers what, and what excitement and curiosity are inherent and which can or need to be added/acquired/experienced extrinsically.
How many associations will we be able to make if AIs will do everything without human intervention?
The Hallucinations are becoming less, significantly by now.
It also might be already were it is cheaper for one of the big few companies to spend millions and billions to teach the LLM / creating the training data necessary for an LLM to do something which it is not yet good enough due to the fact, that they sell this capability then to everyone who wants to use this capabilitiy.
We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
I'm 'vibing' on the side a handfull of small things, no LLM trained on particular what i'm asking to do. Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The field of mathematics went from 'useless' to 'you start better using it' to 'gamechanger' in how fast? 1 year after coding? less?
I want signes that we hit a real problem, instead I get cheaper tokens, Chinese models becoming very good as open models, new model updates from the others, mathematicans now saying how good it is etc.
Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
If AI makes us 10% or 20% betteer, which is not that much, this alone will lead to companies reduing their expensive staff by 10-20%, which will has real impact on a job area. Some jobs are already hard to sell like cyber security and basic image tasks.
Yes? What is the mega-solid technique that is used for it? Armies of people using curated data and reviewing it by hand? That is exactly one of my points: shifting the work elsewhere for specialized tasks. More replicable, improved, but, it scales infinitely and is autonomous? Can you assert that?
I am not denying there is some use (a lot of uses!) for this, but this is more nuanced than just: oh, they will replace us. Not at all, that day, with the current technology, is not going to arrive. This is just a systematization, fitting and tweaking of human knowledge by curated data. It is not the one true superintelligence they are selling us. To begin with, they do not have a concept of truth, but of probabilistic truth. Only that poses already a very, very big problem for the path to perfection.
> We have not seen the end of Reinforcement Learning, which does need a lot less training data but more compute.
Noone said the opposite, but I would like to know at which cost and if it is feasible. We do not have even enough compute power for current technology.
> . Its very capable of stringing together enough things so it can clearly follow handwavy things i tell it to do, analyse error messages, analysing screenshots etc. all by itself.
I use it every day for these tasks and it works well BECAUSE I review the output and makes me go faster. It finds a lot of things I would have not found and it also hallucinates another handful of them, which confirms my point about AIs not being able to be fully autonomous in any future point in time unless tweaked exactly for the task, and even then, it can still miss judgement a human could have for edge cases. So I am not sure of how bad or good it can be compared to a human but I am pretty sure it cannot be more reliable than an expert in many situations.
> Chinese models becoming very good as open models
I think they will be better in the long term if they follow this path. Not absolutely better but when mixing with economics and the fact that no frontier model is totally reliable anyway... why pay a lot for something that needs human inspection anyway?
> There is not a single real ceilling in sight, we only have clear barriers like compute but constant fast progress.
The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time. If it becomes generically knowledgeable and reliable, it is bc of data fed into it and curated and tweaked by humans. This is not an original idea from myself, there are armies of people doing this every day around the world, you can check. This is where a lot of improvement comes from. Can this be reused? Of course. It is a generic solution? No way.
> Only half a year ago I had to babysit an LLM, now i tell it 1-3 sentences and it just goes and does it. And that stuff runs without compile errors etc.
Yes, I also do one-off scripts like this and code snippets, even reviews and others. Now go design a full distributed system. Use agents if you want. We come back in six months and compare it to a system that was properly written and tested by humans and we can compare the quality on some grounds:
1. how long it takes to add new features?
2. which ones act more according to spec once added?
3. when adding features, which ones have more bugs?
4. in the face of an error, will the agent delete my whole AWS infra (count the money losses if possible also)?
5. will I understand (or need to understand, but I bet yes) this code at some point in the future?
Not easy at all.
I would say yes and it will scale. It will either happen through central LLM just paying for it and scaling it up to everyone on the planet (literaly) OR by the agentic layer every big business is building into their systems.
You needed some human to use your tool optimized for their company? With agentic layer you no longer need this. And if you look at companies like Google, they were pushing this notion for ages already because they saw an adoption problem of more 'complex' tools and trying to make it simpler and easier. Now you can act from the other side too.
> We do not have even enough compute power for current technology.
Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
> The ceiling is the paradigm itself, as I mentioned above. There is not a single chance with current technology that something could become "generically knowledgeable" and "reliable" both at the same time.
It doesn't need to be perfect, it only needs to be better than the avg human. And the current LLMs are already better than aat least 1-2 people in my team.
> Now add the human factor of requirements and back and forth
Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
And all of this can be copy and pasted. There is only one harness needed which becomes the expert security reviewer and tomorrow everyone can have it.
I'm still discusing progress with LLMs with people and still not everyone is using it or playing around with harnesses or developgn an agentic layer. We still have a lot of work to do to even see how good it will become while it already is really good.
People are already borred of AI today and making wrong decisions based on the current level of AI while i think we will see continues progress for years.
So you mean AI will be useful for any general job without lots of training for those jobs? How about new tasks? Tasks it has not been tweaked for. When I deviated from the average, and not really weird things, when programming, the output was way worse than average stuff. And this is an explicit target of AIs nowadays.
I think you are missing a lot of details here, honestly.
> Exactly. Right? There is no ceiling if its clear that we don't even have the hrdware. But the hardware is a bottleneck not a ceiling.
No, the hardware is a bottleneck, the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks (which is useful). Which is exactly the opposite of what singularity and AGI have been promising.
The systems we have now (unless the paradigm changes) will keep doing, essentially, fitting. No concept of truth and limited inference. That inference is based on already existing data, not on future data. In fact, there have been experiments about feeding output back to the input of LLMs and the degradation of the quality is very visible. If they are supposed to be so "intelligent", why it happens?
> It doesn't need to be perfect
I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
> Yeah for now. Grill me skill made it a lot easier. Harness engineering is also being worked on, agentic layer, ai factories etc.
I will not deny there could be progress, but nothing similar to "autonomous", "reliable", "super intelligence" or "singularity" with this paradigm.
In fact, often in my experience, this is a waste of tokens for subpar results that shift the technical debt elsewhere. I mean if you try to develop full systems by "vibe-code like" techniques. If you use them judiciously, you can accelerate your workflow, maybe 2x, but not much beyond that if you want to have something worth to be used. Note that here I am talking about the full thing: with testing, quality, maintenance concerns and everything together.
If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way. Unless your job is convincing enough people that you can go 10x faster every time, deliver some sub-par thing, and find another customer, which, to me, would equal a scam.
I'm pretty sure we will solve this issue. Either already through World Models or another architecture.
It could also be, that we just need a 10 or 100 Trillion Parameter model to match so many generic ways of solving tasks and keeping the concept in the LLMs 'head' to solve it that it will just emerge with parameter size. Like with fable they said that it can chain together exploits which wouldn't work as standalone exploits.
What if the only real barrier is the depth of understanding of concepts and this is exactly what is getting solved with parameter count?
But look how young this field really is if you start counting it when it became relevant on mass. Its not 'just' an LLM which is changing the world, its machine learning overall. Robotics wouldn't be were it is today if its not for machine learning. Took humans time and energy to take the leap, to start learning what the status quo is and then actually doing more with it.
> the paradigm as we know it is a ceiling unless you massively and continuously feed this system with average tasks
While I do think its doable to achieve AGI in 5-15 years, even if it doesn't happen and it always means that people train an LLM or whatever, if you need 10 experts to teach this to an LLM OR every single senior has to teach this to their juniors every single time, the LLM will always win.
I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
the craziest thing about this? If i wouldn't tell them what they are doing wrong, they wouldn't even know it.
Quality is already a very flexible term for a lot of people.
> I can agree that for lots of tasks it does not. But for others it is just not a tool good enough.
Yet.
> If you want to ship a sub-par thing that will go to the rubbish in a couple of months, then yes, you can do that. But that will fail commercially any way.
Now we come to the reality: I have seen so much garbage software its crazy. People using md5 as a password hash in 2024! No clue what coding best practices are, teams without code review, teams without a security expert not even knowing what crazy things they do day in day out.
Just a few month ago a team build an API for my team including a Swagger UI. Half of it didn't work. You pressed a button on the Swagger UI and a 500 returned.
And do'nt underestimate what it means that a lot of business people don't like software people. You know that fruit basket we get? and water and stuff? they don't do it because they like us they do it because thats what you have to do. if a Product Owner starts vibe coding with AI, he will have leadership convinved in no time, then it goes on production and it will run for waaaaaay longer than anyone would have guest.
Besides that there is plenty of software were complexity is less relevant or security is not that big of an issue.
Please elaborate. How? With which technique? Currently the only path forward is to feed more data and tweak for specific situations (fitting, basically). How does that help in the general case or in new situations with current tecchnology (LLMs, concretely). Noatter how far you get, this is not a general or reliable solution. It van only simulate more generality or more reliability by training and tweaking. Nothing else. At least, with this paradigm.
This does not mean they will not be useful. What I challenge here is the AGI or singularity. We are far from that.
> I'm now team lead for 10 years and every single year I teach them the same thing over and over and over again.
I have been a lead and an architect also for years at different position. I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of. And if it is, then you have to dumo so much context that it is better to go do it yourself. There is a cost to that also actually. It is not just so "dry and technical" the knowledge. Maybe yes to learn Java patterns or C++ constructors or the like.
But not for "given this situation with all these specifics", which solution would you bet on? Probably the LLM will give you a shitty REST API that is not what u need at all.So u tell the AI. It gives u something else generati g 30-50% of "decorated code". Now it seems to workso you use it. Now you do this every day. Come back in 2 months. You generated a lot of fat.
Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns. But I saw some and use a prompt with limited access and the best I can take out for my speed + control when coding is tech discussions to decide on it, error catching, test generation, one-off scripts... But never "make an app like this or that". If I ever do that (I did it a couple of times) is for scaffolding and later throw away 70%.
Namely, to see something that runs on screen quickly. But later you need to spend time yourself as usual. Not a bad thing, just that this is not what you deliver and need the work done. Iterations etc.
Reinforcement learning can just solve things even if they are new. It doesn't understand how a tool works? Give it a vm with the tool, a thousand agents and let it discover it automatically.
Use the thumbs up/down emoji + chat analysis when a customer is unhappy, feed that to a RL Loop.
The AI Researchers though work on World Models, grounding the AI and letting it simulate. It can do the simulation in parallel (unlimited) and choose what is best.
> I think you miss how much tacit knowledge and judgement there is inside the brains of each of us that an LLM is not capable of.
But thats my problem. Soooo many do not have this even as senior developers.
> Now you have a bug. You do not know even where to start. Thisis theprice to payfor speed, as usual: technical debt.
Yeah now i just ask the LLM to describe to me the bug. Works very well.
> Now you tell me you put three agents to talk and burn 2000 usd in tokens. Great! Is the final solution better than what you would have achieved? Not sure at all.
This is the thing. It only needs to make the team 10-30% better to compensate token budget with one work collegue. We have reached this level in my opinion already. Choosing a head count vs. choosing tokens.
But it becomes cheaper and easier and better. So you will not just be able to do ith with 3 agents but with 20, 50 or 100.
It will be better if your team is an avg team. It will be worse if you have a high profile team, for now. But man our industry has such a weird broad quality spectrum.
> TBH I am not into agents bc I do not trust a tool sniffing all my code and for copyright concerns
In worst case, my team always do code reviews, I do a code review on an ai instead of a human and adjust the harness or the infos the ai can access. I can actually work on making this workflow better and then i can clone it or spin it up for every single PR. For a human? I have to train them and they might leave.
But there are plenty of cases were code doesn't matter. Researchers write a lot of random shitty uggly code as long as it does what it does, it doesn't matter. I have scripts for small tasks, we have microservices which do one thing because it is a tech stack we only need for one use case.
I think you are confusing giving theories about what a bug might be with certainty. It does help bc it csn accelerste things, but many times I had AIs with challenging bugs throwing a lot of misleading theories to me. For the easier bugs, I was just as capable most of the time. Not every time, so there is some potential time saving there. But also time waste.
As for research and fast prototyping you are right: I find it a good tool to explore bc yiu do not need the quality of a final product and researxh is in big part throwaway work.
But I was talking about software that needs features, maintenance, etc. This is just not the same thing.
We see AI factories going in this direction but there is no real 'the open source ai platform' thingy.
It needs connectors to integrate with k8s, hyperscalers etc. it needs to be able to have a basic router, a way of configuring expert agents and interaction options for the human in the loop.
There is for sure things we need to build or change, but it def feels like to me that it would immediadly fix a few things today.
I'm not disappointed that it doesn't advance as fast as it feels
- The AI researcher might be working for a lab or company with much less funds than the top dogs. Are they likely to discover something that is worth it before a bigger model becomes more capable?
Just because a AI will be able to do it in the future does not mean that us plebs will be allowed to have access to it. That alone is enough of a reason for smaller labs to keep going; having a seat at the table.
I have a meta thought..
Hypothetically what happens once there is no more data to be fed to the system? Are we expecting AI to invent its own data and reach full cognition?
Currently we are feeding it the data that humans created but if we stop (i.e "why bother?") thinking that AI will do it all?
https://www.utoronto.ca/news/training-ai-machine-generated-t...
In practice you can do it a bit (generated data from a better / different model is fine, some generated data might be useful if there is non generated data etc.)
Sure it can, turn up the "temperature" a bit.
There's this notion that human "jumping" is magic. It's not. It's all based on inputs. Including unrelated inputs, past inputs, and feeding yourself your own thoughts.
The hard part is not the ability to make conceptual jumps. That's just random search. The hard part is discrimination: whether a given mental jump is "creative" or "insane". Iterated, the problem is that of balancing between the two failure modes: relax your thinking too much, and you'll start thinking nonsense thoughts; tighten it too much, and you'll be just following immediate-term rewards and obvious thought trains. It takes time to find that balance, and plenty of people at various points err in one or the other direction (e.g. small kids in particular tend to err on the "crazy non-sequitur side", but that's because they're learning the basics of reality and social interactions).
That's what they mean by LLM's can't jump. They mean it can't make a creative jump. Their example is Einstein's Theory of Relativity - It's not a random jump.
Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
14% used to feel about right for me - I'd spend the rest of the time researching approaches and libraries, planning things out in issues, or sometimes just thinking really hard about problems I ran into.
Now... I still do those things, but I'm doing many of them faster - and I'm often doing them while my coding agents are churning away on code.
There's also this weird effect where the harder a problem is the more I can get done in parallel with it, because an agent might need to spend 20 minutes on it without my involvement.
The cost of prototyping has crashed. So you can A/B/C... various ideas, test them in the real world or by personal preference, to taste, and converge on designs that work for you.
This used to very hard and expensive. Now it's so cheap it makes the idea -> test -> refine loop much tighter and faster.
Oh yes! I would think "I wonder how the data would look if I did X" and feel the pain of complicated SQL... but then the LLM would just do it and I got to see if I liked or not! It was a delight
> Anyone else finding they're spending more time writing code (or at least driving agents to write code) now?
Not really, as once it is time to write code, the problem has been defined/understood (to the degree possible with knowledge acquired at the time), and encoding it is largely an exercise in typing along with verifying assumptions via test suites.
Does GenAI quicken some portions of the above workflow? Sure, in the same way IDEs with contextual code snippet suggestions can make encoding faster.
I don't review code that either works or doesn't - most HTML and CSS layout code for example. There I test it on desktop and mobile and commit it if it works.
Ditto for stuff that's simple. A JSON endpoint that runs a SQL query and returns some JSON? If it works and a glance at the tests looks OK then I trust my agents wrote it properly.
I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
I think this is THE issue of our time as programmers to be honest: do you review every line of code an agent writes?
An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
That is *exactly* the sort of area I *wouldn’t* blindly trust AI, there’s a huge security boundary there. What if the AI is doing string concatenation with user-provided data???
But you could be defensive with a security checklist in agents.md and have adversarial review, if you wanted.
It's actually getting worse due to "AI code bloat", for example I have 16k lines of code to review across 3 apps by the end of this week. Normally it would be a quarter of that, but what Claude produces is extremely verbose in some places and anemic in others, and I can't tell at a glance what's right and what looks right with that much ground to cover.
We don’t because everyone is accountable for his or her own mistakes. So everyone is incentivized for their recklessness to not be the root cause of some bug.
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you ever asked your users? What about bug reports? Is the amount and rate decreasing?
Good example of what not to review if you're working on your hobbies. Also exploratory can sometimes be done this way. However, this ultimately boils down to how you approach programming as an engineering discipline, including your responsibility for the outcome.
> I'm getting more confident with my judgement over what needs a close look and what doesn't over time, as so far I haven't been majorly burned my any mistakes that snuck through.
This doesn't generalize well. If you drink raw milk, or if you don't wear your seatbelt, or if you don't escape your user input correctly, you'll probably be fine, but I really hope aspiring programmers/engineers don't take this attitude towards any serious task. One should always examine their biases, tools' failure modes, etc. regardless of how many times something didn't fail.
> Honestly, it's similar to being an engineer on a larger team. You don't review every line of code written by every one of your coworkers.
One [should] review the code they're responsible for. In a team, people usually assign you (or ask you) to review code, and the work is divided accordingly. If the code isn’t reviewed by the code owners, it’s a problem, not something inspiring!
> An increasing number of expert programmers are moving in the direction of NOT reviewing every line. It's working out OK for a lot of them.
Have you considered that the sheer amount of code being generated is what makes thorough review infeasible, not that it’s a desirable approach?
("Escaping" user input is not good practice though, use parameters, assuming you are talking about SQL.)
How do you observe the issues that aren’t apparent via a GUI? Do you notice the circular logic in your reasoning?
Have others noticed this as well? This is going so far as to me losing interest in side projects because I have "lost touch" with the code base.
And the parts we don't care about aren't necessarily worse, they are just... arbitrary. Could be good, could be bad, no one knows, because no one really cares.
I find that if I care about something a lot, it's a pretty similar time investment than pre LLMs. And it makes me feel invested and proud in the result, motivated to show it and improve it.
If I care about something very little, in the past I just wouldn't have done it at all. Now I might, but I feel that same disconnect you mentioned.
I think being strategic in what we do and do not care about is likely the key skill we'll have to build to actually make the best of the tech.
They also have an update -- linked from the original study! -- explaining that it's out of date and no longer reliable, and explaining why they had to cancel a follow-up study because it was understating productivity gains (but also was showing wins for the people who carried over from their previous study): https://metr.org/blog/2026-02-24-uplift-update/
The authors of this paper decided to ignore all of METR's follow-up data and discussion, and to report only the ancient number from early 2025 (a time when Windsurf was state of the art). And then, rather than apologizing for it, and caveating it as a number not to be taken seriously, they described it as a study done "recently."
That's either shockingly dishonest or incredibly out-of-touch.
Or the hype machine is even stronger, making people inflate those numbers even more.
So much has changed since late 2025 one can’t really draw any conclusions from this.
In fact, I’m guessing things will continue to move so fast that by the time one were to execute a survey of developers, many of the responses and findings are no longer relevant.
Reminds me of COVID and how everyone was fighting over early trends during that time.
Coding has never been expensive as it is nothing more than a reification of a solution to a problem as it is understood at that time.
It is the underlying understanding of the problem which has always been expensive and remains so.
You can improve some parts with LLM (make more prototypes, iterate faster), but if humans are the bottleneck and the problem itself is complex it will not be 10x improvement.
Of course, maybe we talk about different type of project, there are also the type of projects like "a web-shop that sells stuff" that someone considered "expensive to code".
My experience is that users (customers) even when presented with a prototype (done fast by AI, or previously, in the slow way) can take days to "discover" or "think" about things they wish. Then more days to understand why some wishes can't be done because it breaks other stuff they want.
Maybe you have smarter, faster, more experienced and decided users. Then I can only say: good for you, you are lucky!
This conflates two orthogonal concerns; understanding what needs to be done with encoding a solution for what needs to be done. I am reminded of an axiom I have long held:
When making software, remember that it is a snapshot of
your understanding of the problem. It states to all,
including your future-self, your approach, clarity, and
appropriateness of the solution for the problem at hand.
Choose your statements wisely.
I disagree with the proposition that being able to generate "wrong code faster" has anything to do with understanding what ultimately needs to be done. In fact, exploratory efforts having the intent to refine problem domain understanding (a.k.a. "proof of concept" projects) is an established industry technique whose work product is best discarded.
But also, no, because they write:
> “coding” (not including bug fixing, testing, etc.)
What if bug fixing includes "coding", or "writing code", or however one would want to define that? Especially in the enterprise setting they evoke, a lot of work will not be "coding" in the sense of churning out new features, but "coding" in the sense of fixing bugs. I know a lot of my "coding" is in this category. But we're not given a number for it. I suspect the slice would be bigger if they included this type of "coding".
Writing code is indeed the bottleneck for same resource constrained companies.
Rapid code development creates more opportunities for trial and error, providing companies with more information for decision making, that previously might have been addressed by meetings.
Of course, this might bring other problems, but it might not right to generally speaking that writing code is not a bottleneck.
The only solid objection to “AI is going replace developers” is “AI is an accelerant.” It helps developers move faster. I haven’t seen anywhere it has fully replaced developers.
Whether this leads to a large number of job losses depends on whether you think we can increase software output by the same factor as the acceleration and still be profitable. I think we can, latent software demand is extremely high. I also think we’re nearing the limit of capability with current models.
Situation could change if more advanced models emerge, but some of the more foreseeable advances probably have compute requirements beyond today’s hardware.
The only point that still holds is that organizational policies and procedures that automate AI use and lower the barrier to entry are more efficient than leaving it up to each individual. Every other point they make is either stale or was never true to begin with.
They quote a study in which developers report to spend 11-14% of their day coding. The rest is stuff like solution design and meetings. The insinuation is that AI can at most automate 14% of your day.
The problem with this argument is that once you have code, some (not all) of the precursors to code go away.
Show me the evidence that AI has an impact on productivity when doing design work. Or reducing meeting load.
My own experience is that AI doesn't tighten the design cycle, and in fact might extend it by encouraging gold plating.
This isn't exactly novel territory, here, Simon. Let's not pretend I'm asking for something strange, unprecedented, or unreasonable.
I have similar charts across my three main open source projects:
https://github.com/simonw/datasette/graphs/code-frequency
https://github.com/simonw/llm/graphs/code-frequency
https://github.com/simonw/sqlite-utils/graphs/code-frequency
More code written is not a good measure of productivity. It could be garbage, or redundant code, or simply not addressing the real or more pressing needs, it could be building the wrong thing, etc.
Like TFA mentions, it's been known for decades LoC is a misleading metric for productivity. It's one of the lessons of software engineering.
In my specific case, lines of code for my published open source projects is a metric that I trust, because I have high standards for those. I have plenty of other projects where I'll accept poor quality, unreviewed code (almost all of https://github.com/simonw/tools for example), but Datasette, sqlite-utils and LLM are not that.
Of course, that's only useful for me personally and for people who trust me to stick to my own self-declared high standards!
Really now?
Maybe because there is none.
One of the main quality of good codebase is simplicity. Which is about how easy for someone else to understand the code. It’s hard to define what simplicity looks like, so the best bet is to avoid the other side, making the code too complex.
And you can make the code complex by shortening variable name, doing code golfing with quicks of the platforms, so smaller LoC. You can also go the other way and increase the LoC by adding unneeded abstractions, repeating slices of code,… There’s a window where the LoC is perfect to attain simplicity, but that amount is an effect of striving for simplicity, not a cause of it. And it’s variable for every problem.
So you got something where the correct value is a different for each case. And trying to manipulate it artificially often results in complex code. And you want to say that is a good metric for productivity?
And in the cases of your projects, there are a lot more info could share that are interesting, like the amount of issues (reported or found by you) that are tied to implementation bugs (coding, libraries API breakage,…) or design issues (requirements conflicts,…), documentation improvement,… Anything that is tied to the actual usefulness of the projects, and not fumbling around with code.
Why though? The only entities who would be interested in such an implication is the AI marketing.
No one real cares about more or less lines of code. But everyone cares about decreasing the complexity of the implementation. Sometimes that means more LOC and sometimes it means less.
I don't know about easy. It's a hard earned lesson of software engineering, backed by research (some of it cited in TFA). Also, there's a related lesson: the more LoC to solve a given problem, the more bugs. That's also a finding backed by research.
I'll be reading your article when you write it, but I confess I'm skeptical.
The AI companies have a vested interest in using this metric, since it's easy to measure (the reason LoC were used even before AI) and there's no doubt that LLMs are writing tons of code. This makes me doubly skeptical.
The OP claims AI accelerates non-coding parts of the job, too, and so the article is misguided.
I ask for evidence.
In response you give me... code output metrics?
That's why Microsoft did a study (referenced in the article), where they measured the time spent on things, so they could get to the truth of how much time people spend coding using "a shape of evidence would you find convincing"
So yes "literally everyone has access to it" (what a revelation) but you still need to be smart in how you use it
In the case of an LLM generated demo, usually deceptively so.
> and give an impression that the design decision has been made.
In the case of a vibed design, this is the opposite of useful for the team.
Well, I expect when you've vibed too much and lost track of the code, and can't answer questions in meetings anymore, you'll stop getting invited to them.
People vibe code because they have no clue about any of that. Not because they're slow typers.
I am working on a compiler for an OCaml inspired language (I am on a sabbatical) and used them a lot in the early phases to check my ideas and search for prior work. I use them less now (mostly debugging at this point), but have made use of LLMs to write some pieces like source map generation, a couple of small and well known algorithms that were new to me, and an analyzer to dump a schema from a typescript module. Other times I've tried to use it for other things, I have ended up regretting it because of subtle bugs
I don't think it made the design phase any shorter but it definitely helped save me energy in researching and made that period feel less like a fever dream. Design phases can be draining for me so that was welcome
That meeting that you spent an hour in to understand the requirements? You don't need that meeting if you're not writing the code. That sync up with the QA engineer you did to hand it off to them? Don't need that meeting if you're not writing the code. That half hour you spent installing vim extensions? Don't need 'em if you don't open vim anymore.
There are engineers whose jobs go well beyond coding, of course. Staff engineers and principal engineers have had their jobs radically change because of AI, but not because it's writing all their code.
But there are also a lot of engineers -- your standard mid-level engineer, or even senior engineers at a lot of orgs with title inflation -- whose job is almost entirely about delivering code, and who spend all day either writing code or engaging in scaffolding around code-writing activities. Let's not pretend that automating away that code writing is a 15% boost.
How are you going to prompt the LLM or validate its output if you don't understand the requirements?
So an 180 from the current coding-automation frenzy..
> Your PM can write the ticket, and your QA can test the output.
If your PM can write the ticket and your QA can test the output, why not cut out the middleman by having the PM write the coding agent prompt(s)?
And if your PM can write the coding agent prompt(s), why not cut out the QA group by having the PM write the verification prompt(s)?
And if your PM can write those prompts, why not cut out the PM and have your CSRs write the feature request/verification prompts?
And if your CSRs can write the feature request/verification prompts, why not cut them out and have the organization's customers write the feature request along with acceptance prompts?
And if customers can write those prompts, why would they pay you?
In many ways, the "save costs by LLM-ing everything" mindset is the same logical fallacy companies fell into in the 90's/00's with regard to outsourcing. The end-state was rationalizing outsourcing call centers, then production system support, then software development, then project management, then...
Once the outsourcing raptor came for the accountants (who wrote the cheques) and remaining management (who cashed quarterly bonus cheques), outsourcing became "less appealing."
Talented, experienced devs leaving the field out of disgust would be pretty disruptive.
Crumbling, faulty technical infrastructure with unmanageable heaps of AI slop that no one understands would definitely disrupt businesses.
Skill atrophy, and an entire generation of "developers" that have never actually written code will be disruptive when something breaks and no one knows how to fix it (this isn't a hypothetical, I'm seeing this phenomenon in a lot of large enterprise groups).
The total elimination of novel solutions, new ideas and new approaches to development and the resulting industry wide stagnation won't be disruptive, but will certainly be a drag.
Replace "AI slop" with "legacy code", and you've got basically any large enterprise today.
The timescales are different though: it takes years if not decades to accumulate classic legacy cruft, while LLMs can spew up huge amounts of incomprehensible spaghetti in seconds.
PMs seem pretty easy to replace. In fact, given any single role to fixate on, it's probably replaceable
It’s because you are engaging in first-order thinking. You imagine the immediate result which is that software engineers are replaced.
If you engaged in second-order thinking you would realize that if software engineers can be replaced, _any_ work can eventually be automated. So why stress about it, we’ll figure out a replacement for capitalism. It seems like the world is starting the slow swing to the left so eventually the problem will be solved.
Of course might be those engineers you mention are engaged in no thinking.
You don't need that meeting if you're not writing the code.
My mates are not, their code editor are not either : I am.
Yes, I could be a clown and be accountable for something I know nothing about. I choose not to.
Not everybody is a cruft-engineer .
It's like all those AI meeting tools. Full transcripts of entire meetings and no human ever takes a 2nd look at them for literally any reason, ever. My favorite is the meeting summaries that nobody reads, and then 2 weeks later a miracle happens and it does get read, only for the AI to summarize nothing useful, or what happens even more often, for the summary to just be blatantly wrong and it implies the opposite of what was said during the meeting!
|Contract|Product|Design|Coding|Testing|Deploy|
Writing Code Isn't the Bottleneck, until writing code is the bottleneck, until it's not again.
That includes talking to vendors, meetings with every layer of stakeholder when just one person digs their heels, etc.
That is truly the final frontier for "AI", and one that it will likely never cross. That would be when even the execs and upper management feel threatened by "AI". But, since they also delegate so much, you often see someone at the bottom of the totem pole in those meetings. This is why nobody is getting replaced by "AI". We really need to move this discussion away from the scifi stupidity already. There is no singularity or godlike AGI about to take over the world.
I hate to use awful terms like "synergy" and "teamwork", but they do have a lot more substance and truth to them than any perceived threat from "AI".
It's a huge piece of shit and if I wasn't forced to use it at work I would never use it.
It writes dumb, throw-away code and adds thousands of dollars/developer in costs. All this crazy code that we're adding to our projects will come back to bite us in the future, there's no way it won't.
I think this is missing an important detail. Lots of time was spent on non-coding stuff, because coding used to be more committal and hence expensive. With how quickly one can code up a quick prototype or even production-ready code these days, the code becomes the communication tool as well.
how come lines of code (or expressions) by an engineer aren't a good way to measure progress (Gates point etc) but GenAI tokens must count and be paid for?
This feels about right as an average across project cycles and different types of companies, and is the same type of number I've suggested here before.
The obvious conclusion is that even if AI reduced coding time to zero, then it would only reduce software development time by that 14%.
Of course AI may be used for other aspects of the job as well as coding, but on the flip side any serious use of AI requires a human in the loop to give work to the AI, steer the AI, assess the output, etc.
It's interesting that this 14% figure is close to Uber's choice to limit AI spend to 10% of developers salary. I wonder where that Uber number came from?
Another rather startling datapoint on the perceived value of AI comes from Microsoft who are also adopting budgeted AI usage, and are looking for "outcomes that move the needle". Their budget guidelines apparently refer to (current? targeted?) per-developer AI usage of "hundreds of dollars a month to a few thousand dollars in tokens".
https://www.techradar.com/pro/tokenmaxxing-is-not-what-we-ar...
For those more junior - keep in mind that a lot of the maximalist rhetoric are from people either selling models, or the cottage industry of people selling you courses or tools to help you use the models. Try and keep in mind software is not a mature industry, it's an immature one, and it's prone to hype and fads.
and why did this study (performed in China I might add) find that women and “mature age” (which is not defined in the study) have less usage of AI tools?
>>>? We suggest a new barrier: using technology to assist task completion signals a lack of competence to perform the task independently.
So basically the study suggests that older engineers and women are reluctant (and especially women) to use AI tooling because they feel they are being judged more on non-technical competencies.
This study may have a strong cultural influence but I would say that one thing they noted I’ve also seen. 41% of engineers in the study had used AI 12 months after the initial rollout. This aligns with my observations. Some people are struggling figuring out how to adopt in their day to day while others are full in.
- In biz development, a dev usually spends 30-40% time on coding, and more time on requirement discussion, integration testing (especially when the tests involves mobilephone or car)
- coding time can be reduced to 30%, which means reduce 20%-30% time of the full pipeline
- meanwhile, every phase and role is using LLM now, for example, product manager can produce longer requirement doc easily (we can use LLM to read it anyway:) Meeting sometimes is more than before, because more document output leads to more reading and discussion.
- I hope to find new ways to express biz requirements, in a more efficient and automatic manner.
- Shorten the requirement-dev-test-deploy loop is very important. OUTPUT is not OUTCOME. It is equal when we can see the final result, instead of intermediate metric.
- Agentic infra is extremely useful, or every one will find a way to access the database, report and ops system, in some weird fragile method.
For me it changes in a way that i would like to have a 24/7 workspace vm setup outside of my work laptop for keeping it running if it wants and looking at it remotely if i want.
The workspace thing would also allow it to have more permissions like downloading, configuring and using headless chrome instead of highjacking my chrome session.
Instead of searching some linux issue, i will prompt claude to generate a small analyser script for checking wha tlinux i have, i will tell it what hardware i have and it fixed my issue in like 5 minutes? That would have been a lot longer before.
A great alternative future to look forward to - 1 employee expected to do the work of 10-100. Those others fired to save money.
that's such a weird metric, why exclude bug fixing and testing? depending on the phase of the project I might spend 100% of my coding time bug fixing
Basically as a senior tech lead in a large company engineering org, I don't have the bandwidth to individually validate every assertion from engineers on other teams OR from every product manager that comes with a half-baked ask. In the past I would be limited by the influence I could get through human relationships to strong SMEs with good judgment, and those folks always thin out as a company grows and calcifies. The number of creative and innovative thinkers dwindles, and the number of people protecting their turf and doing the minimum not to get fired increases. As a result many good ideas can get blocked by random gatekeeprs with poor imagination, poor expertise or both. However with AI I can follow up on gut instincts and fact check a lot more things, and ask incisive questions that can cut through a lot of organizational bullshit.
That's where I think most of the AI gains are today. Of course once AI plateaus and normalizes I think it will be baked into the org structures of tomorrow. But for now it offers real competitive advantage to those with the expertise to ask the right questions.
This is a depressing stat. The real productivity gains come from leaving soul sucking big tech companies where nothing gets done with any sort of urgency.
It's easy for a small team to adjust workflows and roles, but I just imagine the office politics must be a waking nightmare in big organisations right now.
Top leadership and/or investors believe in the benefits of AI.
Thus, skeptics stay silent. Who wants to loudly contradict their bosses' boss?
In practice, I do not see a big shift yet in workflows that would require a lot of politics. We just have the agent implement the code, and then it still goes through the usual code review and QA processes.
Only now with more effective models and harnesses do many developers realize how good these tools are at investigating bugs, etc.
Before the price-decreased GPT 5.6 Luna we barely had access to enough cheap AI to last for a month of work.
LLMs from Anthropic and co we get through a central tool which buys api tokens from Azure and AWS.
Its surprisingly fast, probably thefastest i have seen technology getting rolled out.
Im more worried about the small/middle sized companies which are software companies but don't get that they are. You know the companies 100% depending on IT but not having the right or good or enough people who often in interviews struggle with basics like code review.
No, they don't! It's easy to dispel myths when the myths are built on straw men. Dumb article.
I must be a crap developer, because I probably spend twice as much time bugfixing and testing than "coding". (Both of which actually involve coding stuff, so I really don't like that distinction they make)
This is stuff AI can be really good at, so brushing that part under the table distorts the picture.
Having said that, I do agree with most of the myths they present.
LLMs are also quite good in writing unit tests and understanding bugs a lot faster than I do, now.
Just a few month back i looked at some yaml stuff for like 20 minutes, played around with it, looked at formatting etc. then i asked the LLM, it immediadly told me what was wrong. I was just blind to that particular wrong char.
Logic bugs? Yeah it can find them too.
A year ago feels like forever
Maybe not 10x, but there's definitely a significant boost in velocity of development and delivery.
We used to be a team of 20+, now only 9, and we are delivering more than before. Besides that, we did some major refactors that were sitting in the backlog for months.
I stopped reading after this. AI has massively impacted most aspects of my non-coding work including the mentioned planning, understanding legacy code bases, setting up environments, etc etc.
Either this article is written by people with skill issues or - given the platform - its a biased and protectionist take that will fall quickly under the march of reality.
But I'd push back on one thing the article implies that AI is automatically a productivity win. It's not. Some days I've shipped two months of work in a few days with AI. Other days, like today, I've burned a whole day and gotten almost nothing done because the proper research was not done by me or multiple agents.
The bottleneck for AI can be the human understanding of how to optimally use the tool. While the bottleneck for the human can be not maximizing multiple agents, or the input the user enters, then the retention of the output. If the user's input is lost, the output falters. If the user doesn't understand what the AI output is, there is going to be a problem eventually.
The article touches on adoption barriers (Myth 7), but it doesn't really get into the ego piece. There's still a wave of experienced devs who either refuse to adopt AI, or use it quietly and don't share what they're doing. That slows the whole team's learning curve. At this point, I think it's pretty much understood that you should be using AI as a dev — not to replace your skills, but to accelerate them. That means still learning new languages, still writing code, still troubleshooting. The tools change, but the craft doesn't.
I think the article is right that the real leverage is organizational, not individual. The teams that succeed with AI aren't the ones giving everyone a license — they're the ones rethinking how they review, test, and maintain code.
What I'm still uncertain about is how to measure whether AI is actually making systems better, not just faster. Lines of code is clearly a bad metric, but I haven't seen a good alternative yet. What metrics are people actually using that feel meaningful?