Id wager my life savings that devs aren’t even 1.5x more productive using these tools.
FWIW I generally treat the AI as a pair programmer. It does most of the typing and I ask it why it did this? Is that the most idiomatic way of doing it? That seems hacky. Did you consider edge case foo? Oh wait let's call it a BarWidget not a FooWidget - rename everything in all other code/tests/make/doc files Etc etc.
I save a lot of time typing boilerplate, and I find myself more willing (and a lot less grumpy!!!) to bin a load of things I've been working on but then realise is the wrong approach or if the requirements change (in the past I might try to modify something I'd been working on for a week rather than start from scratch again, with AI there is zero activation energy to start again the right way). Thats super valuable in my mind.
Because the world is still filled with problems that would once have been on the wrong side of the is it worth your time matrix ( https://xkcd.com/1205/ )
There are all sorts of things that I, personally, should have automated long ago that I threw at claud to do for me. What was the cost to me? Prompt and a code review.
Meanwhile, on larger tasks an LLM deeply integrated into my IDE has been a boon. Having an internal debate on how to solve a problem, try both, write a test, prove out what is going to be better. Pair program, function by function with your LLM, treat it like a jr dev who can type faster than you if you give it clear instructions. I think you will be shocked at how quickly you can massively scale up your productivity.
I also primarily write Elixir, and I have found most Agents are only capable of writing small pieces well. More complicated asks tend to produce unnecessarily complicated solutions, ones that may “work,” on the surface, but don’t hold up in practice. I’ve seen a large increase in small bugs with more AI coding assistance.
When I write code, I want to write it and forget about it. As a result, I’ve written a LOT of code which has gone on to work for years without touching it. The amount of time I spent writing it is inconsequential in every sense. I personally have not found AI capable of producing code like that (yet, as all things, that could change).
Does AI help with some stuff? Sure. I always forget common patterns in Terraform because I don’t often have to use it. Writing some initial resources and asking it to “make it normal,” is helpful. That does save time. Asking it to write a gen server correctly, is an act of self-harm because it fundamentally does not understand concurrency in Erlang/BEAM/OTP. It very much looks like it does, but it 100% does not.
tldr; I think the ease of use of AI can cause us to over produce and as a result we miss the forest for the trees.
It excels at this, and if you have it deeply integrated into your workflow and IDE/dev env the loop should feel more like pair programing, like tennis, than it should feel like its doing everything for you.
> I also primarily write Elixir,
I would also venture that it has less to do with the language (it is a factor) and more to do with what you are working on. Domain will matter in terms of sample size (code) and understanding (language to support). There could be 1000s of examples in its training data of what you want, but if no one wrote a commment that accurately describes what that does...
> I think the ease of use of AI can cause us to over produce and as a result we miss the forest for the trees.
This is spot on. I stopped thinking of it as "AI" and started thinking of it as "power tools". Useful, and like a power tool you should be cautious because there is danger there... It isnt smart, it's not doing anything that isnt in its training data, but there is a lot there, everything, and it can do some basic synthesis.
I used to complain when my friends and family gave me ideas for something they wanted or needed help with because I was just too tired to do it after a day's work. Now I can sit next to them and we can pair program an entire idea in an evening.
I think that's what is missing from the conversation. It doesn't make developers faster, nor better, but it can automate what some devs detest and feel burned out having to write and for those devs it is a big win.
If you can productively code 40 hours a week with AI and only 30 hours a week without AI then the AI doesn't have to be as good, just close to as good.
Last night I was working on trying to find a correlation between some malicious users we had found and information we could glean from our internet traffic and I was able to crunch a ton of data automatically without having to do it myself. I had a hunch but it made it verifiable and then I was able to use the queries it had used to verify myself. Saved me probably 4 or 5 hours and I was able to wash the dishes.