The AI Situation in Software Development
srikanth.ch
srikanth.ch
Recently I asked Claude (Fable) to use multiple threads to speed up a computation that could take several seconds to run while the user was waiting. Instead, it found a way to start the computation earlier in the background while the user was doing other things, so that it would be finished by the time the user was ready.
More features is always good, right? Let's build dropbox+gmail+netlify+spotify+youtube+hackernews+... \s
I think it's not an option. The benefits are just too large for me.
>I do know we will never go back to mainly programming through code again, that’s for sure.
I think, in some niches, i.e. where there's something not well represented in the training set, it still makes sense to write code by hand. But I am not sure that it will continue.
In all aspects there will be dinosaurs and deniers and there will be embracers.
And one does not memorize algorithms.
This thinking was popularized by coding competions which existed before leetcode and inspired leetcode. Schools themselves produced people who believed these puzzles are what makes you superior developer - one of us, special and choosen.
Say who?
stragnely i dont miss that feeling and kind of dread going back to it
It's not type-safe, not object oriented, not functional. Has poor tools to highlight syntax or navigate through "wordbase", doesn't fail fast. It has no tests and has too large room for machine or other humans to interpret it.
Very often it's easier for me to express my thoughts in Java, which is ironically known to be a "wordy" language. But it's nowhere close to wordiness of English.
So I don’t think AI will be much different.
almost every inference operator either has ZDR or an opt out from training
unless you think they're just lying and training on business users data
All the loops and agents don’t protect you from generating garbage.
Which sucks because then how are you supposed to improve your skills when you’re just getting the answers all day… answers you can’t verify?
People are more confident than they ought to be. Always have been. But AI throws gas on that fire.
But if the person that is above me forces me to use the latest craze tool to do *my* job then that is no longer a place for me to stay.
Now, it’s at the point where it’s like running a development team of very eager amnesiacs. I’ve found the trick is exhaustive documentation by a lead agent, and then having a fresh agent work as a coordinator across as many subtasks as the project sensibly allows. This way the individual components stay on spec, as does the ultimate integration. It’s only really this year that this workflow has started to actually function, and it still needs human supervision - but less and less over time.
I give it two years, tops, and everyone everywhere is building bespoke software because it’s trivially easy.
This point seems lost on a lot of principals. I’ve had very little success with these grandiose designs and change requests from RFCs/specs. The context windows just can’t keep it all together and very quickly the approach unravels.
I posit that the further ICs were from writing code at this point in their career, the more they suffer from AI psychosis. It’s the same ivory tower they were already on, just a different order they’re giving.
Just wondering if the professional programmers here are finding the same thing? Or different?
Questionable output, generally shunned by artisans.
But possibly good enough for some.
It's like how product people / C levels have absolutely no understanding of what makes for good code or a good engineering shop (aside from perceived costs.)
That said, I'd really like to see data to compare which approach works the best.
Even with preplanning and post hoc analysis thrown in, I am seeing way more than 2x return on my investment. Where are your numbers coming from?
I wonder how that progress is being measured. Lines of code or counts of PRs? Sure... but I thought the matter of measuring productivity by lines of code was already well-understood as being misguided.
I'm having trouble reconciling all that supposed productivity with the real world where software isn't getting better, delivered faster, or becoming cheaper - unlike virtually all breakthroughs in industrialization (printing press, weaving loom, etc) which led to a quick increase in at least one of such factors.
I'm not denying that AI helps with and excels at parts of the software development lifecycle, but from my experience those parts overall contribute to a small increase in output or merely shift the work elsewhere (where it may just not be part of whatever measurement is being used).
It sucks, but you don't usually have the time to pour over code when you manage multiple engineers either, so you have to learn how to do thorough but targeted reviews, minimize distraction, maximize efficiency, etc. A lot of these skills transfer over to managing agents.
We've only had truly decent agents capable of running long-horizon tasks for less than a year, I think it's worth calibrating around that: it's too soon to expect the entire industry to visibly shift.
That said, every senior engineer I know has gone all-in on agentic development, and juniors I mentor are getting a lot done as well.
With juniors it's important to make them understand that these models can't be blindly trusted and the output needs to constantly be critically evaluated.
But engineers who know exactly what they are doing have really been able to make some awesome things this year. I'm also working on a few really cool things, more than before, more ambitious as well, without sacrificing quality or craftsmanship.
I can also seem where some trends are headed. The breadth of software available to both harm and help you is going to explode, and computing is going to look a lot different soon. I'm already building targeted health apps for myself, bespoke personal apps and tooling, development tools, I'm working on games, libraries, various kinds of research, you name it. It feels like an intellectual Renaissance, and within a decade I expect things to look a lot different even if models stopped improving today.
You do have to work differently with these models. Your code evolves in a different way, and testing habits have to adapt. Clients are going to accept less stable but more ambitious demos. Prototyping and research have suddenly become very cheap. We're going to see the effects of the spread through STEM and the arts.
I was talking more about end-to-end feature/product development process from the perspective of the business, and not merely the "writing code" part. Things like figuring out what to build, what code to write (which remains - just that now you are writing a prompt instead of writing the code directly), design, customer support, regulatory compliance/etc.
From this perspective I believe that even if AI does actually automate away software development, we will find out that on average it was never actually the bottleneck nor a significant cost of the product lifecycle. Thus I'm not in a hurry to go all-in on AI just because I don't see old-school human-powered SW dev at being the bottleneck, at least not on the products/projects I work on (large software products with established customer bases).
In a lot of cases I find that the what to build is the biggest bottleneck - and in fact the relative slowness and occasional pushback (because they have skin in the game - see below) of conventional human-powered SW dev forces the stakeholder to think really well about what they want and gives them time to refine that idea. If I were to give them a hypothetical "SW dev in a minute" magic wand it would result in a lot of ill-defined & incoherent features being thrown at the wall which will quickly overwhelm support, destroy their reputation with customers (or worse, regulators) and become a perpetual maintenance burden slowing down any further development to a standstill (whether human or agentic). So I don't do that for my own sanity, even though that magic wand already exists and it's called Claude Code.
This is not to say I find AI completely useless - I see plenty of opportunities for AI to help out with non-product-related tasks - housekeeping that doesn't introduce/change any functionality and would normally rot in the backlog forever are good candidates, assuming I have good end-to-end tests and a full isolated environment where the agent can drive the whole product to QA its own work as to not create additional review burden (otherwise, I would have to review and QA it myself which is often as much effort as just doing the work myself).
> you don't usually have the time to pour over code when you manage multiple engineers either
One thing that I get with humans but not agents is skin in the game and self-preservation. A human writing code will write in such a way as to minimize future work to himself next time there's an outage, support ticket or likely change request and 2) will retain memory of what he wrote so that he will be able to address that support ticket or outage much quicker than a human having to effectively reverse-engineer the code to figure it out. From that perspective, agentic development didn't save time, it merely shifted it from the development phase to when the first support ticket or problem arises (the former can be scheduled and budgeted in, the latter is worse as it arises at unexpected times, derailing whatever else is happening).
(this obviously only works in environments where engineers are encouraged to own things end-to-end. I know some places treat their human engineers as a dumb one-shot ticket->code translator and discourage thinking or taking responsibility beyond that. These are the same places that wonder why they're not able to make any progress because they're stuck endlessly fire-fighting the crap such a degenerate process produces)
> With juniors [...] output needs to constantly be critically evaluated
This raises a scary catch-22. How are the juniors meant to evaluate the output without the expertise they'd only gain by being "in the trenches" for 10+ years? LLMs only help to a point - as they can be convinced of anything depending on prompting or persuasion (I've had Claude adamantly claim things that were wrong which I only picked up on because of actual experience. Similarly, I can easily steer it off the correct path with just a couple suggestions).
The problem of upskilling engineers is not new - billions have been thrown at the problem in the form of bootcamps/etc and yet there's still no good replacement for actual experience doing things and getting burnt in the process. I'm worried that deference to AI will mean a sea of perpetual juniors (but with senior responsibility and blast radius) and the jobs market for actually skilled & experienced talent drying up even more.
> bespoke personal apps and tooling, development tools, I'm working on games, libraries, various kinds of research, you name it.
Absolutely, same here. But building goodies for yourself is different from building them for paying customers. The latter brings a certain expectation of stability and support. In fact this is why even pre-LLM, there is a huge gap between being able to program and being able to launch a product, and why many programmers' pet projects never made it to the latter.
> Clients are going to accept less stable but more ambitious demos.
Demos? Sure. But generally speaking they're paying you for the actual implementation and subsequent support and maintenance of it. Salespeople being able to bang out a demo with a few prompts without engineering involvement could also mean more cases of them selling an impossible feature (that human-powered engineering could've warned them about).