I would have zero tolerance if anyone that did anything less in my orgs. Owning and understanding the code you submit is the one job that matters most now.
"But we passed SOC2 and that is all our customers care about" they say, right up until they are in the headlines for a data breach directly resulting from them putting all customer data in plain text internet facing databases the whole team can access from their workstations. What did they think was going to happen? Survivors bias is a hell of a drug.
It's a whole other level of junk. 100x worse than now. So multiple what you just said, x100.
sure, students and juniors shoot themselves in the dick if they let LLMs do everything, but they're fucked anyway so it doesn't matter,
Why don’t we just care about results? Code quality, performance, clarity are all measurable. Why bother with philosophical positions on these?
Seriously, this was a huge thing in C and C++, where you had to know/remember/know to look up very specific error codes and conditions in an API and if you didn't then it was very much considered to be a skill issue. This was maybe 15 years ago.
It was not unless you were part of some very weird circle. There was always manuals and other references that listed everything about a platform or library. Then we got ‘man’ and ‘info’. And I remember being able to download the full docs of windows’ various documentation.
That’s like saying as a writer you were looked down at because you had to look at a dictionary every once in a while.
And they were correct. You're entirely right that this time it isn't different, but in the sense that the quality of both developers and code has been going downhill for decades. The only thing that has kept this industry alive is that hardware got more performant faster than developers became more inept.
The non-strawman version of Socrates warning, that writing-based cultures would lose memory, take recorded fact at face value and mistake knowledge for wisdom was true as well. Your post is an example of this, you read something about Socrates, you didn't investigate and were fine with the appearance of knowledge, and because books can't defend themselves you mistook that for understanding.
That's what he was concerned about, hordes of people with a vague assemblage of facts but no genuine capacity to critically think.
is there a lot of evidence for this? You can broadly put technology into two camps. Technologically that, as it becomes more powerful, demands more attention and skill from the user and just the opposite. A bow is more powerful than a stick but also more demanding to use. A car is more powerful than your feet but your population ends up obese. Almost every single piece of software, with the exception of maybe debuggers, who an astonishingly amount of seasoned developers do not utilize (I suspect for that reason), is a car and not a bow.
There's a study on English college students roughly ten years ago who were tasked to read the first few paragraphs of Dicken's Bleak House. They were even allowed to use a dictionary and the internet. A majority could not interpret the text. That's where we were at before LLMs. My rural working-class grandfather at an underfunded Catholic school had to translate Ovid from Latin into German in 8th grade. It wasn't even a personal attack, we're just this fried now.
What I was asking was is: If it's true as you say that we have replaced our atrophied skills with other ones, what are some of those, or technologies that facilitate them? Be that governmental, social or otherwise. Because we have a decline in reading, a decline in physical ability, a decline in sociality, to the topic of the thread a decline in the quality of software and engineers, but very little to show for it in my book.
Writing, definitely. Reading, maybe. Thinking is another thing. I can think about code in a lot of ways just by reading a description of the code, or knowing how I directed the code to be written, etc. I think people are thinking about the code differently.
the current pace of LLM advancement only has to keep for five more years for handmade code to be considered a liability. I wouldn't worry about being able to recall obscure APIs or whiteboard leetcode. I'm far more worried about having to change careers at 40.
Number and order of arguments, long names, etc. are just bureaucracy. Knowing them doesn't mean you understand the codebase any better than if you rely on the IDE for it.
It's a bit like remembering the exact date an historical event happened vs. knowing at which point in history it did, what other events relate to it, etc. Neither makes you better or worse at knowing and understanding the history of the event.
We need to continue the tradition of elevating humans through mentoring or we won’t have a future scaffolding for work to be done.
I think a possible solution on both fronts (maintaining software quality, preventing skill atrophy) may be implementation of strict token budgets. Make the individual programmer choose how they are going to spend their limited tokens for the day/week. Provide just enough to use it as a plan/development/review tool, but nowhere near enough to just go full send AI vibe coding software factory slop mode all day long.
They could be committing plan files alongside any nontrivial change to application code, and the plan file can have everything you'd care about, like the invariants it held to and why, the rejected ideas and why, the acceptable risks and why, etc.
This is solved by better usage of LLMs, not less.
The divide to me isn't engineering skill vs those who lost it, it's people who care vs. those who don't care. It's a higher level thing now that we're working at a higher level (delegation).
We survived. Some upsides, some downsides, life continues.
They made choices that made them dumber is what I'm saying.
Yet programmers will still know commands. Abstractions can make things easier but it doesn't require losing skills.
I call it an "abstraction" because that's what it is. Instead of you writing commands manually into the terminal, you use a program that calls it for you and does the hard work. That's what an abstraction is, you're abstracting away something by making an interface around it.
All interfaces, as a matter of fact, are abstractions.
Programmers just moved up a level , not dumber, we are now managers of a team of 'agent' programmers. The deliverable is now a functionality instead of a specific block of code
If I need to understand a specific line of code it means I did something wrong in planning or in requirements for testing.
What I’m describing is thorough documentation of requirements (acceptance criteria, if you like), and then encouraging Claude to be agile in execution.
As long as the outcome is well-defined, it is expected and normal to iterate on implementation.
But will Claude give you an authentic rationale and a traceable, verifiable "line of reasoning" for those things? Or will it just construct the next plausible Markov chain built on whatever Reddit thread it ingested at random?
You can ask Claude or any LLM for citations, and it will RAG them out ex post facto. Those actually aren't citations, they're just web searches for related articles, and they don't necessarily support the assertions that you're asking to cite.
I am sure that Claude and the others can produce intermediate logs of their inference and "reasoning" process while they are processing stuff, but can they really go back within the context window and construct an authentic apologia for a specific thing when you ask for it?
If you ask it 3 times to generate 3 verifiable reports that confirm its claims, will it answer with the same process and same answers each time?
See, when you ask a human to justify a result or a decision, they can often do this very meticulously. If a judge writes a decision from the bench, or a firefighter describes how his battalion knocked down an apartment fire, or a systems admin describes how he configured a NAS, they will all be relying on their training, and precedent, and specifications, and things like that, and they can give you reproducible results and solid justifications for the way they did things. When mistakes are made, and money or life is lost, they can be accountable and you can modify that process to set a precedent for the future.
But Claude? How in the world will it produce the same results twice? It is non-deterministic. That is the fundamental issue of LLMs and genAI today. They are all non-deterministic, and SWE treat them as if they are somehow reliable, or produce reproducible results, or that they can follow a procedure or a specification, outlined in their prompts and context, and produce results.
No, they only produce results by accident and happenstance, and they only justify them ex post facto by making things up. There is no humanity or deterministic activity in an LLM. You'll never verify "why" they chose that string of tokens, because they could've easily chosen a very different stream of tokens. In fact, now with watermarking, the most deterministic thing will be hitting that watermark standard at all costs!
Specifically, humans are known to decide subconsciously, then invent some "reasoning" out of thin air to justify it.
This matches my experience with decision-making in software projects.
Dementia patients also do this.
No, it will invent retroactively a plausible sounding reason why someone might have done it that way. These are very different things.
The fact that some course of action was previously mentioned in a reasoning trace, or any other context, makes it more likely to be performed. It has nothing to do with the reason that it was mentioned in the reasoning trace.
And it's not "no relation", it was brought up as an attempt to fix/subset the original claim.
And compilers made every software engineer worse at writing Assembly and better at skills higher up the stack. This will be no different.
With compilers, you still had to think in depth about the code, write it, develop and maintain muscle memory, understand what you were doing.
Many are not doing any of that, and coding is a skill you need to constantly practice.
I do.
Most people sucked at programming before AI and they still suck after it. Who cares what "many" do. We as individuals can demand higher bars from ourselves and our teams. AI can increase that bar or lower it depending on how it is used.
And then after I replied, edited your comment to change that context a bit.
So as for ypur original assertion, in this subthread, no... not even remotely the same.
An example: 'read input from CLI, do this specific transform on it, send log with this text to external service, then output transformed data to CLI' is a program. Going up the abstraction levels from assembly to Ruby, you always know the "What" of the program, while you understand less of the "How" at each step. This is not the case with LLMs used in the common way. The problem with LLMs is that you no longer have a grasp of the "What", because you are not reading the code; you may have memorized the ambiguous English "What" that I wrote above, but that is not sufficient to understand the true behavior because English is not sufficient to represent it; it must be represented in a programming language. (What is 'read'? What is 'send'? What happens with errors?) By the time you have sufficiently represented the program in English to be nonambiguous, you've written more text than the code would have been, with just as much rigor.
So LLMs represent a fundamental difference to traditional abstractions: the user has lost the understanding of what is actually happening, because they are prompting ambiguously, and thus they cannot possibly have a grasp of the behavior, because the behavior is not defined in their prompt. In other words, they are programming in a new programming language that has a ton of undefined behavior that just randomly happens. No one could understand the What of their program when using such a language, unless they compiled it to a nonambiguous language and then understood that instead.
Of course it is possible to read and understand the code output by the LLM. It is even possible to build a whole knowledge graph in your head, as you traditionally would, and have a pretty full understanding of the whole thing. But, you are always doing this with the outputted code, not with the inputted prompts. The prompts are not analogous to a higher level of abstraction in this way; they are analogous to JIRA tickets. You don't have to read the assembly to understand your program, but you do have to read the LLM outputs.
The other issue is that the above paragraph's process of understanding is discouraged. The time taken to fully understand the outputs approaches the time it would have taken to write them yourself, so is very rarely done, and that increasing rarity of understanding is what we are concerned about. The programmers are moving from an understanding of the program into an understanding of the JIRA tickets that produced the program; they are becoming product managers. The thing is, currently you still need someone who understands the program beyond the level of a product manager in order to retain maintainability and quality. Maybe in the future you won't, but right now you do.
Your first prompt could be instructions to guide creation of a spec, which leads to a test suite you personally validate, which leads to code to pass those tests in a deterministic way. Then it is your job to review and QA it until it is quality enough to submit to a peer for code review.
There are many many ways to use these tools and carefully monitor and quality control what they write like a good engineering lead might, even if they do not type much code themselves at work. The job of the most senior engineers often ends up being heavily biased towards reviewing code all day and maintaining quality control. This is already how things were before AI.
Torvalds reviews a -lot- of code and writes very little now.
Also, the staff engineers who review code all day typically do not really understand the program either. They understand the spec and they have a model of how things should work such that they can answer "should" questions, but they cannot tell you what exactly is going on in the depths of the program; that's not their job. The tools are not capable right now of creating a quality program that the operator only understands at a conceptual level. Again, someone has to actually know, still.
Regardless, I run a security auditing company where we constantly find severe bugs in human written code proving the engineers writing it had major gaps in their understanding of their own code. And, others have found flaws like that in my own hand written code! We humans are often very blind at seeing the flaws in our own logic and AIs learned from us.
Also, as code reviewers that can spot bugs authors and AIs miss, we are often paid a lot more than the original authors for our time. Also it almost never takes as much time as writing the code in the first place which likely required many round trips with customers to establish needs and fix bugs etc.
Your claims imply that only someone that hand types every semicolon is capable of understanding the code does not hold up even in the pre AI world.
Code review was the most important job in software engineering before AI and that is still true now.
Make that some/a lot, but not all.
There is a clear bifurcation. A lot of those that never cared really don't care now, but then you get others that can use it to crank that much harder than ever before.
If you put the same brain effort in but leverage LLMs you really can be dramatically more productive.
Just like when Boeing decided that their QA process should be abridged to save wasted time and money!
And that worked perfectly.
(edit: rereading I think maybe you don't support not reviewing all of LLM code you submit, but the above was written with another impression)
If you care primarily about the quality of the end result, as opposed to being attached to a particular process of achieving it, then you're happy about how things have gone.
And I've seen exactly this across multiple large and small organizations in multiple countries, in addition to being all over here and the red site.
This is quite distinct from enjoyment.
> And I've seen exactly this across multiple large and small organizations in multiple countries, in addition to being all over here and the red site.
I know that experience, but I do have a different opinion on this:
Many such people studied computer science (a similar point holds for mathematics, physics and some engineering disciplines) because they love coding.
Unluckily, the job that they got does involve barely any clever coding, but rather
- tolerating and gluing together shitty code pieces that someone else wrote
- tolerating red tape
So, such people simply do in their free time what they love. And to come back to my point "similar point holds for mathematics, physics and some engineering disciplines": these people try to use their little free time to prove some novel theorems or put their shed full of machines like 3D printers, CNC mills, CNC lathes, laser cutters, self-built pick-and-place machines, ... to work on their own beloved engineering projects.
We must be living on different planets, because from everything I can see, software quality is at a historic low.
I do think LLM's are making some genuine programmers less intelligent, but I think this type of person makes up a decent portion of its users too
Sorry to say this, but he doesn't sound like a nice guy.
People judge me for less all the time and I usually don't care.
> People judge me for less all the time and I usually don't care.
Maybe its a case of "takes one to know one", but isn't this literally what a sociopath is? You act without caring what people think. I don't usually like to say that about a person I don't know but clearly you won't be affected by that.
I don't care when people are wrong about me and my moral compass is telling me what is right. My moral compass is mostly centered around love and respect for others. If someone misunderstands that about me, they are within their rights to do so.
I have love and respect for your friend even though we've never met and you've given me full indication he is a money grubbing sociopath. And for you, who doubles down into saying your description is not sociopathic.
If everything you're saying is true, your moral compass is clearly not aligning with your actions here. That's fine, genuinely wanting to love and respect others and making a mistake in your actions on that moral compass is a one time mistake that can be easily remedied. The alternative is that you're actions align, the mistake was your conception of your moral compass, which would be a far bigger problem.
I'm a person who is over-reliant on LLMs, doesn't know how most of the things in the codebase work, and is atrophying both in programming and in general cognition.
I used to care, a lot. And I still do.
Are you prioritizing other things? Do you think losing your cognition and programming skills won't have a negative effect on your project?
Those seem bad but maybe they fit into a bigger picture for you?
When I recently found a small bug in how NTP is implemented in Home Assistant OS that was affecting my use of it, I thought about just reporting that bug.
But I set the bot forth to work on corrective measures, instead.
The first proposed fixes were overbearing and awful. Maybe they'd have worked but it hurt my soul to look at them. I couldn't accept them -- not for my system at home nor for anyone else's.
Through a laborious process (that many here could have skipped entirely), I eventually got what I feel is a cromulent fix down to a few lines of changes in an init script.
I didn't write it. All I did was direct (and sometimes brow-beat) the bot, rejecting things over and over while optimizing at a high level until it produced an acceptable result.
I now have a fix that is tidy-enough that it can be applied to running systems. It is easy to understand what it does, and why it does it -- even for me. Perhaps I should submit a PR...
> They just don't care. They didn't before either.
...but this kind of blanket mentality really puts a damper on my willingness to do so.