How AI assistance impacts the formation of coding skills

My hypothesis is that the AI users gained less in coding skill, but improved in spec/requirement writing skills.

But there’s no data, so it’s just my speculation. Intuitively, I think AI is shifting entry level programmers to focus on expressing requirements clearly, which may not be all that bad of a thing.

That's not what the study says. It says that most users reflect your statement while there is a smaller % that benefits and learns more and faster.

Generalizations are extremely dangerous.

What the article says simply reflect that most people don't care that much and default to the path of least resistance, which is common every day knowledge, but we very well know this does not apply to everyone.

I wonder if we're going to have a future where the juniors never gain the skills and experience to work well by themselves, and instead become entirely reliant on AI, assuming that's the only way

This is up there with believing tobacco companies health "research" from the 30s, 40s, 50s, 60s, 70s, 80s, and 90s.

If you sucked before using AI you are going to suck with AI. The compounded problem there is that you won't see just how bad you suck at what you do, because AI will obscure your perspective through its output, like an echo chamber of stupid. You are just going to suck much faster and feel better about it. Think of it as steroids for Dunning-Kruger.

https://www.youtube.com/shorts/0LeJ6xn35gc

https://www.youtube.com/shorts/vXecG_KajLI

(I jest a bit, actually agree since turning assembly->compiled code is a tighter problem space than requirements in natural language->code)

IF I was totally dependent on it, I would be in trouble. Fortunately I am not.

So you know what do, what I've been doing for about a decade, if the internet goes down? I stop working. And over that time I've worked in many places around the world, developing countries, tropical islands, small huts on remote mountains. And I've lost maybe a day of work because of connectivity issues. I've been deep in a rainforest during a monsoon and still had 4g connection.

If Anthropic goes down I can switch to Gemini. If I run out of credits (people use credits? I only use a monthly subscription) then I can find enough free credits around to get some basic work done. Increasingly, I could run a local model that would be good enough for some things and that'll become even better in the future. So no, I don't think these are any kind of valid arguments. Everyone relies on online services for their work these days, for banking, messaging, office work, etc. If there's some kind of catastrophe that breaks this, we're all screwed, not just the coders who rely on LLMs.

This is why I suggest developers use the free time they gain back writing documentation for their software (preferably in your own words not just AI slop), reading official docs, sharpening your sword, learning design patterns more thoroughly. The more you know about the code / how to code, the more you can guide the model to pick a better route for a solution.

What happens when github goes down. You shrug and take a long lunch.

You are at least a decade late to post fears about developers reliance on the internet. It was complete well before the LLM era

i would work on the hundreds of non-coding tasks that i need to do. or just not work?

what do you do when github actions goes down?

Your pizza restaurant is all wonderful and all but what happens when the continual supply of power to the freezer breaks? How will you run your restaurant then?

* all services are run at a loss and they increase price to the point the corp doesn’t want to pay for everyone any more.

* it turns out that our chats are used for corporate espionage and the corps get spooked and cut access

* some dispute between EU and US happens and they cut our access.

The solution’s having EU and local models.

Actually, the last thing you probably want is somebody reverting back to doing things the way we did them 20 years ago and creating a big mess. Much easier to just declare an outage and deal with it properly according to some emergency plan (you do have one, right?).

CI/CD are relatively new actually. I remember doing that stuff by hand. I.e. I compiled our system on my Desktop system, created a zip file, and then me and our operations department would use an ISDN line to upload the zip file to the server and "deploy" it by unzipping it and restarting the server. That's only 23 years ago. We had a Hudson server somewhere but it had no access to our customer infrastructure. There was no cloud.

I can still do that stuff if I need to (and I sometimes do ;-) ). But I wouldn't dream of messing with a modern production setup like that. We have CI/CD for a reason. What if CI/CD were to break? I'd fix it rather than adding to the problem by panicking and doing things manually.

You're an engineer, your goal is to figure stuff out using the best tools in front of you

Humans are resilient, they reliably perform (and throw great parties) in all sorts of chaotic conditions. Perhaps the thing that separates us most from AI is our ability to bring out our best selves when baseline conditions worsen

I'll happily optimize my life for 99.999% of the time.

If the Internet is down for a long time, I've got bigger problems anyway. Like finding food.

One of my advantages(?) when it comes to using AI is that I've been the "debugger of last resort" for other people's code for over 20 years now. I've found and fixed compiler code generation bugs that were breaking application code. I'm used to working in teams and to delegating lots of code creation to teammates.

And frankly, I've reached a point where I don't want to be an expert in the JavaScript ORM of the month. It will fall out of fashion in 2 years anyway. And if it suddenly breaks in old code, I'll learn what I need to fix it. In the meantime, I need to know enough to code review it, and to thoroughly understand any potential security issues. That's it. Similarly, I just had Claude convert a bunch of Rust projects from anyhow to miette, and I definitely couldn't pass a quiz on miette. I'm OK with this.

I still develop deep expertise in brand new stuff, but I do so strategically. Does it offer a lot of leverage? Will people still be using it on greenfield projects next year? Then I'm going to learn it.

So at the current state of tech, Claude basically allows me to spend my learning strategically. I know the basics cold, and I learn the new stuff that matters.

Well, yeah. You were still (presumably) debugging the code you did write in the higher level language.

The linked article makes it very clear that the largest decline was in problem solving (debugging). The juniors starting with AI today are most definitely not going to do that problem-solving on their own.

I'd kinda like to see this measured. It's obviously not the assembly that matters for nine-9s of jobs. (I used assembly language exactly one time in my career, and that was three lines of inline in 2003.) But you develop a certain set of problem-solving skills when you code assembly. I speculate, like with most problem-solving skills, it has an impact on your overall ability and performance. Put another way, I assert nobody is worse for having learned it, so the only remaining question is, is it neutral?

> everyone's acting like all human coders are better than all AI's

I feel like the sentiment here on HN is that LLMs are better than all novices. But human coders with actual logical and architectural skills are better than LLMs. Even the super-duper AI enthusiasts talk about controlling hoards of LLMs doing their bidding--not the other way around.

I could have sworn I was meant to be shipping all this time...

One take-away for us from that viewpoint was that knowledge in fact is more important than the lines of code in the repo. We'd rather lose the source code than the knowledge of our workers, so to speak.

Another point is that when you use consultants, you get lines of codes, whereas the consultancy company ends up with the knowledge!

... And so on.

So, I wholeheartedly agree that programming is learning!

People naturally try to use what they've learned but sometimes end up making things more complicated than they really needed to be. It's a regular problem even excluding the people intentionally over-complicating things for their resume to get higher paying jobs.

They can give plausible architecture but most of the time it’s not usable if you’re starting from scratch.

When you design the system, you’re an architect not a coder, so I see no difference between handing the design to agents or other developers, you’ve done the heavy lifting.

In that perspective, I find LLMs quite useful for learning. But instead of coding, I find myself in long sessions back and forth to ask questions, requesting examples, sequence diagrams .. etc to visualise the final product.

The amount of context switching in my day-to-day work has become insane. There's this culture of “everyone should be able to do everything” (within reason, sure), but in practice it means a data scientist is expected to touch infra code if needed.

Underneath it all is an unspoken assumption that people will just lean on LLMs to make this work.

And then you find out someone else had already solved it. So might as well use the Google 2.0 aka ChatGPT.

For hobby projects though, it's awesome. It just really struggles to do things right in the big codebase at work.

My first thought was that I can abstract what I wrote yesterday, which was a variation of what I built over the previous week. My second thought was a physiological response of fear that today is going to be a hard hyper focus day full of frustration, and that the coding agents that built this will not be able to build a modular, clean abstraction. That was followed by weighing whether it is better to have multiple one off solutions, or to manually create the abstraction myself.

I agree with you 100 percent that the poor performance of models like GPT 4 introduced some kind of regularization in the human in loop coding process.

Nonetheless, we live in a world of competition, and the people who develop techniques that give them an edge will succeed. There is a video about the evolution of technique in the high jump, the Western Roll, the Straddle Technique, and finally the Fosbury Flop. Using coding agents will be like this too.

I am working with 150 GB of time series data. There are certain pain points that need to be mitigated. For example, a different LLM model has to be coerced into analyzing or working with the data from a completely different approach in order to validate. That means instead of being 4x faster, each iteration is 4x faster, and it needs to be done twice, so it still is only 2x faster. I burned $400 in tokens in January. This cannot be good for the environment.

Timezone handling always has to be validated manually. Every exploration of the data is a train and test split. Here is the thing that hurts the most. The AI coding agents always show the top test results, not the test results of the top train results. Rather than tell me a model has no significant results, it will hide that and only present the winning outliers, which is misleading and, like the OP research suggests, very dangerous.

A lot of people are going to get burned before the techniques to mitigate this are developed.

Overfitting has always been a problem when working with data. Just because the barrier of entry for time series work is much lower does not mean that people developing the skill, whether using old school tools like ARIMA manually or having AI do the work, escape the problem of overfitting. The models will always show the happy, successful looking results.

Just like calculators are used when teaching higher math at the secondary level so basic arithmetic does not slow the process of learning math skills, AI will be used in teaching too. What we are doing is confusing techniques that have not been developed yet with not being able to acquire skills. I wrack and challenge my brain every day solving these problems. As millions of other software engineers do as well, the patterns will emerge and later become the skills taught in schools.

> "We collect self-reported familiarity with AI coding tools, but we do not actually measure differences in prompting techniques."

Many people drive cars without being able to explain how cars work. Or use devices like that. Or interact with people who's thinking they can't explain. Society works like that, it is functional, does not work by full understanding. We need to develop the functional part not the full understanding part. We can write C without knowing the machine code.

You can often recognize a wrong note without being able to play the piece, spot a logical fallacy without being able to construct the valid argument yourself, catch a translation error with much less fluency than producing the translation would require. We need discriminative competence, not generative.

For years I maintained a library for formatting dates and numbers (prices, ints, ids, phones), it was a pile of regex but I maintained hundreds of test cases for each type of parsing. And as new edge cases appeared, I added them to my tests, and iterated to keep the score high. I don't fully understand my own library, it emerged by scar accumulation. I mean, yes I can explain any line, but why these regexes in this order is a data dependent explanation I don't have anymore, all my edits run in loop with tests and my PRs are sent only when the score is good.

Correctness was never grounded in understanding the implementation. Correctness was grounded in the test suite.

> AI assistance produces significant productivity gains across professional domains, particularly for novice workers. Yet how this assistance affects the development of skills required to effectively supervise AI remains unclear. Novice workers who rely heavily on AI to complete unfamiliar tasks may compromise their own skill acquisition in the process. We conduct randomized experiments to study how developers gained mastery of a new asynchronous programming library with and without the assistance of AI. We find that AI use impairs conceptual understanding, code reading, and debugging abilities, without delivering significant efficiency gains on average. Participants who fully delegated coding tasks showed some productivity improvements, but at the cost of learning the library. We identify six distinct AI interaction patterns, three of which involve cognitive engagement and preserve learning outcomes even when participants receive AI assistance. Our findings suggest that AI-enhanced productivity is not a shortcut to competence and AI assistance should be carefully adopted into workflows to preserve skill formation -- particularly in safety-critical domains.

But that's what "impairs learning" means.

How exactly? Do you tell the agent "please write a test for this" or do you also feed it some form of spec to describe what the tested thing is expected to do? And do these tests ever fail?

Asking because the first option essentially just sets the bugs in stone.

Wouldn't it make sense to do it the other way around? You write the test, let the AI generate the code? The test essentially represents the spec and if the AI produces sth which passes all your tests but is still not what you want, then you have a test hole.

Sometimes I wonder if people who make statements like this have ever actually casually browsed Twitter or reddit or even attempted a "large" application themselves with SOTA models.

I can iterate on entire approaches in the same amount of time it would have taken to explore a single concept before.

But AI is an amplifier of human intent- I want a code base that’s maintainable, scalable, etc., and that’s a different than YOLO vibe coding. Vibe engineering, maybe.

you can also read human code and empathise what were they thinking while writing it

AI code is not for humans, it is just a stream of tokens that do something, you need to build skills to empirically verify that it does what you think it does, but it is pointless to "reason" about it.

Yes, we can use it 10,000 times to refine our recipes, but "did we learn from it"? I am doubtful about that, given that even after running with the same prompt 10 times, it will give different answers in 8/10 responses.

But I am very confident that I can learn by iterating and printing designs on a 3D printer.

I’m starting to believe that people who think AI-generated code is garbage actually don’t know how to code.

I hit about 10 years of coding experience right before AI hit the scene, which I guess makes me lucky. I know, with high confidence, what I want my code to look like, and I make the AI do it. And it does it damn well and damn fast.

I think I sit at a unique point for leveraging AI best. Too junior and you create “working monsters.” Meanwhile, Engineering Managers and Directors treat it like humans, but it’s not AGI yet.

It reduces the context switching between coding and referencing docs quite a bit.

On the contrary, without mastery guiding, AI writes code in the most boilerplate way possible, even if that means compromising logic or functionality.

> … which then introduces cognitive load for anyone who hasn't encountered these patterns previously

And for those who have. This is the enterprise Java effect. The old trope is Java was designed to make all devs median and all produce the same median code so enterprises don't have to worry about the individual devs, it's all the same bowl of unflavored oatmeal.

When you read code from vibe coding novice, it's difficult to grok the intended logic because that's buried within these chunks of enterprise pattern boilerplate as if the solution was somehow regex'd at random from StackOverflow until some random combination happened to pass a similarly randomized bag of tests.

The cognitive load to reverse this mess into clean clear expression of logic is very high whether a human or machine "coded" this way.

In both cases, the antidote is caring for craft and mastery first, with an almost pseudocode clarity in expressing the desired outcome.

OK, but -- even this doesn't guarantee the result one wants.

Because even if the master writes the code themselves, they may find their intent was flawed. They expressed the intent clearly, but their intention wasn't helpful for the outcome needed.

This is where rapid iteration comes in.

A master of software engineering may be able to iterate on intent faster with the LLM typing the code for them than they can type and iterate on their own. With parallel work sessions, they may be able to explore intention space faster to reach the outcome.

Each seasonal improvement in LLM models' ability to avoid implementation errors while iterating this way makes the software developer with mastery but lack of perfect pre-visualization of intent more productive. Less time cleaning novice coding errors, more cycles per hour iterating the design in their head.

This type of productivity gain has been meaningful for this type of developer.

At the same time, the "chain of thought" or "reasoning" loops being built into the model are reaching into this intention space, covering more of the prompt engineering space for devs with less mastery being unable to express much less iterate intent. This lets vibe "coders" imagine their productivity is improving as well.

If the output of the vibe coder (usually product managers, if you look closely) is considered to be something like a living mockup and not a product, then actual software engineers can take that and add the *-ilities (supportability, maintainability, etc. that the vibe coder has never specified whether vibing or product managing).

Using a vibed prototype can accelerate the transfer of product conception from the PM to the dev team more effectively than PM just yelling at a dev tech lead that the dev hasn't understood what the PM is saying the product should be. Devs can actually help this process by ensuring the product "idea" person is armed with a claude.md to orient the pattern medianizer machine with the below the waterline stuff engineering teams know are 80% of the cost-through-time.

There's not a lot of discussion of prototype vibing being a new way for product owners and engineering teams to gain clarity above the waterline, or whether it's productive. Here's a dirty secret: it's more productive in that it's more protective of the rarer skilset's time. The vibe time wasted is paid by the product owner (hallelujah), the eng team can start with a prototype the product owner iterated with while getting their intent sorted out, so now engineerings iterations shift from intent (PM headspace) to implementation (eng headspace).

Both loops were tightened.

> you run the risk of adding further complexity in the process of 'fixing' the bugs and before you know it you have a big stinking ball of mud.

Iterating where the problem lies, uncoupling these separate intention and iteration loops, addresses this paradox.

I don't think that's true? From what I understand most labs are making money from subscription users (maybe not if you include training costs, but still, they're not selling at a loss).

>(thankfully market dynamics and OSS alternatives will probably stop this but it's not a guarantee, you need like at least six viable firms before you usually see competitive behavior)

OpenAI is very aggressive with the volume of usage you can get from Codex, Google/DeepMind with Gemini. Anthropic reduced the token price with the latest Opus release (4.5).

I am not saying you should be struggling performatively, like a person still proud in 2026 that they are still using Vim for large projects (good for you, eh), but sometimes you need to embrace the discomfort.

You should be concerned if you're outsourcing your work to it, though. There's also no benefit to doing that outside of laziness (the research shows no statistically significant productivity improvement)