Pretty much sums up the LLM fanbase.
I don't think this is realistic. I'm a good programmer, and it speeds up my work a lot, from "make sense of this 10 repo project I haven't worked on recently" to "for this next step I need a vpn multiplexer written in a language I don't use" to, yeah, "this 10k line patch lets me see parts of design space we never could have explored before." I think it's all about understanding the blast radius. Sonetimes a lot of code is helpful, sometimes more like a lot of help proving a fact about one line of code.
Like Simon says, if I'm driving by someone else's project, I don't send the generated pull request, I just file the bug report / repro that would generate it.
but that acceleration is exactly because you're not good at that language
And how do you know if that is the case or the person/team using the LLMs is one of the good ones?
So the safest answer is just "no".
Speed is seductive.
The bar isn't "this is a known good contributor". Its "this is a known good contributor working in a space they have knowledge in and has a track record of actually checking and thinking about LLM output before submitting it." It's much higher and I don't see how you can approve people on an organization-wide basis.
> LLMs make me significantly faster at writing code I was mediocre or bad at. But when I use it to write code in domains I have more knowledge in I see design and correctness problems all over the place and actively fix them and it slows down my output.
i am reminded of this quote: it takes more cleverness to debug code than it takes to write it. if you write code as clever as you can, by definition you are not clever enough to debug it. using LLM makes your code many times more clever than what you could write yourself. which means by the same definition the code is to clever for you to understand or debug it.
The entire problem is that before the meatbag code was either not submitted at all (developer knowing they are not competent enough to do the fix) or the volume of it was low.
With LLM people not competent to even review, let alone write are emboldened to just throw shit on the wall at rapid pace. So the wall is entirely covered by the shit
The problem with this line of thinking is the same with "I so good as C developer, my code is so-safe!".
And we see what reality instead tell: Yes, exist people where this claims are true, not, is not even a decently sized minority.
Whenever I see this arguement, I'm reminded that most programmers don't know what they do for work
You can't review it.
Are you relying on your colleagues to do that, or is this riddled with bugs? Or is it code you're producing for personal use only so it's not worth mentioning as it's not sped your work up, it's just let you write a little play program.
i know what a kernel module is and im reasonably certain that the patch is safe, but there is no way in hell i would have found that solution (i would have given up). in a world without llms, the project would have died.
Time and time again I've had a project (such as a DSL to SQL compiler, automatic Rust codegen, CSS development) stall because the LLM took a short sighted decision.
I later found better solutions by querying Reddit and upon consulting the LLM, it basically said "oh shit I'm sorry"
It's honestly pretty arrogant to tell a senior engineer that you "really hope" they've gone over some code. AI generated or otherwise.
I really hope usted checked your code
At this point I'm pretty sure I did the homework for people in college who are now senior engineers
This is commonplace. So commonplace that most have worked “checking the LLM” into their workflow so deeply that essentially all that’s done is prompt followed by a mini code review.
To suggest a senior engineer blindly accepts modifications without code review kinda hints at you not using LLMs to realize how quickly it will make a mess of things if you don’t hold it’s hand.
This whole comment is incoherent. And you're the one "hoping" people check things?
Normally I dislike AI slop in comments, but in this case I think it would be an improvement.
Now it's totally possible that they're an awful developer (who knows!), but it's arrogant to assume that with no evidence.
And I agree, some of the worst devs I've worked with have been PhD's or had otherwise impressive credentials (ostensibly). And I absolutely think at least 1 of them were just lying about their backgrounds.
we have 2 very high value DAU, one of whom is me, and probably will max out at 1000 in our wildest dreams.
long term, our biggest concern is a security regression that lets outsiders see our internal information
Why are they often so desperate to lie and non-consensually harass others with their vibing rather than be honest about it? Why do they think they are "helping" with hallucinated rubbish that can't even build?
I use LLMs. It is not difficult to: ethically disclose your use, double check all of your work, ensure things compile without errors, not lie to others, not ask it to generate ten paragraphs of rubbish when the answer is one sentence, and respect the project's guidelines. But for so many people this seems like an impossible task.
Because they can't tell the difference between what the machine is outputting, and what people have built. All they see is the superficial resemblance (long lines of incomprehensbile code) and the reward that the people writing the code have got, and want that reward too.
AI is absolutely terrible for people like that, as it's the perfect enabler.
It's not about helping. It's about the feeling of clout. There are still plenty of people who look at Github profile activity to judge job candiates, etc. What gets measured gets repeated.
I believe that most of the ills of social media would disappear, if we eliminated the "like" and "upvotes" buttons and the view counts. Most open source garbage pull requests may likewise go away if contributions were somehow anonymous.
This is a good thing, it's an opportunity to make open source development processes robust to this kind of sabotage.
Yeah that seems to be their primary use case, if I'm honest. It's possible to use them ethically and responsibly, much in the same way it's possible to write one's own code, and more broadly, do one's own work. Most people however, especially in our current cultural moment and with the perverse incentives our systems have created, are not incentivized to be ethical or responsible: they are incentivized to produce the most code (or most writing, most emails, whatever), and get the widest exposure and attention, for the least effort.
Hence my position from the start: if you can't be bothered to create it, I'm not interested in consuming it.
For example, using AI as an editor. It doesn't write anything for you and you try and avoid suggestions unless you're stuck.
This might not be helped by the fact that there are a lot of seemingly psychotic commenters attacking anything which might have touched an LLM or any generative model at some point. Their slur and expletive filled outbursts make every critical response look bad by vague association.
Having sensible explanations like in TFA for the rules and criticism clearly visible should help. But looking at other similar patterns, I'm not optimistic. And education isn't likely to happen since we're way past any eternal september.
I was a food delivery driver back in the mid 00's to the mid teens. Early on, GPS was rare and expensive, so to do deliveries and do them effectively, you had to be able to read a map and mentally plan out efficient routes from the stochastic flow of orders coming out.
This acted as a natural filter, and "delivery driver" tended to be an interesting class of people, landing somewhere in the neighborhood of "lazy genius". Higher than average intelligence, lower than average motivation.
Then when smartphones exploded in the early 10's, the bar for delivering fell through the floor, and the job became swamped with people who would be best identified as "lazy unintelligent". Anyone who had a smartphone and not much life motivation was now looking to drive around delivering food for easy money.
Not saying the job was ever particularly glamorous, but it did have a natural mental barrier that tech tore down, and the result was exactly as one would predict. That being said, I'm not sure end users noticed much difference.
I have friends who order a lot of DoorDash and UberEats and they complain constantly about how awful the delivery service is.
The problem isn't that they haven't noticed, it's that they keep paying for the terrible service, even as the price goes up.
There are cool people on the other side as well, unfortunately those aren't usually who get assigned unless escalations take place.
Most shops are built based on juniors that need to build enough curriculum to go elsewhere as soon as they get some scars.
Yet not only those projects keep coming, now plenty managers dream about replacing those juniors with agents.
Don't get me wrong, tech is why I am here. But if it works, Alice and Bob don't care one bit about how the product exists.
well, they think they don't. until their pii gets leaked all over the internet because whoops our s3 bucket was publicly accessible, or until the service goes down because whoops our llm deleted the prod db...
If you will forgive an appeal to authority:
The hard thing about building software is deciding what one wants to say, not saying it. No facilitation of expression can give more than marginal gains.
- Fred Brooks, 1986
Management, unsurprisingly deemed those precious. They could email them out anytime, working weekend to fix problems their kind were the cause. Sure sir.
They excel at communication. Perfecting the art.
Now LLMs are there to accelerate the trend.
I'd be more concerned if I was someone who signed up to play ping pong two hours a day and do a bi-weekly commit.
There was a time not so long ago where I was watching "a day in the life of a software engineer" videos on Youtube and I was wondering if some of these were parodies. I still remember one in particular which I'm pretty sure was a parody, but it was only marginally distinguishable from the others.
But submissions into slavery for immediate gain accomplishes little, and costs society a lot more (physical and mental health issues are a huge burden).
Those parodies you saw, they were caricature of elite engineers, who sacrificed decades of his life to become so competent. Can work from home, eat pasta while glancing over a PR and just hit approve.
That you resent the luxury doesn't make it undeserved privilege.
That being said, most people in the privileged positions you described are there by sheer luck and connections. In the very very best-case scenario that offends them the least: they stumbled upon an opportune position and were smart enough to make full use of it... in the first 6 months (when people pay the most attention and lasting impressions are formed). And then rode the reputation they made for years. Their value as engineers on the team after the initial honest burst of productivity becomes... very unclear from that point and on, shall we say.
Again, I've met engineers who fully deserved their privileges. 2-3 times over 24 years of career though (a good chunk of it as a contractor so I've been around). My anecdotal evidence obviously means nothing but we all develop pattern-matching skills with time, making me think what I saw is generally the statistical curve that would apply almost everywhere. Maybe.
True, regardless of that, for sure with LLM we are borrowing Technical debt like never before.
Do you work alone i presume? Everyone now is engineer. In my department, even managers are "writing code". Producing thousand of lines of ansible code, that nobody can review, with multiple lines of doc that nobody will read. It is just a mess.
IT isn't the only one - finance and law had the issue since forever, AFAIK - but now I'd rather be in a field that's _actively repellent_ to them.
Regarding quality overall, I agree, it's truly a cursed field. It was bad before; and with LLMs, going against that tide seems more difficult than ever.
I think this is a very rosy view of programmers, not borne out by history. The people leading the vibe coding charge are programmers, rather than an external group.
I know it's popular to divide the world into the technically-literate and the credulous, but in this case the technical camp is also the one going all in.
This response 1000% was crafted with input from an LLM, or the user spends too much time reading output from llms.
Ray Kroc's genius was to make people forget that you get what you pay for.
Not picking on you in particular, but most of the anti-AI crowd can’t present their case compellingly and have an utter lack of humility.
Do You really want it?
There is also a second face of that: people are lazy. They wouldn't develop their own skills but rather they would off-load tasks to LLM-s, so their communicative abilities will be fade away.
That's looks like a strong dystopia for me.
How is this mutually exclusive with teaching better than most humans? Part of these "corporate" datasets include deep knowledge of the world's best literature and philosophy, for instance. Why can't it be both?
> Do You really want it?
If I'm in a hurry, don't know where to start, or don't have money for someone to teach me—sure.
> There is also a second face of that: people are lazy. They wouldn't develop their own skills but rather they would off-load tasks to LLM-s, so their communicative abilities will be fade away.
This is a recapitulation of the Luddite argument during the Industrial Revolution. And it's valid, but it has consequences for all technological change, not just this one. There was a world before Google, the Web, the Internet, personal computing, and computers. The same argument applies across the board, and the pre-AI / post-AI cutoff looks arbitrary.
Ah, so now we get to the "ed tech" question. What is teaching? Is there a human element to it, and if so, what is it? Or is it something completely inhuman? Or do we need to clarify what meaning of "teaching" we're talking about before we have a discussion?
Part of those datasets also include 4chan.
This wouldn’t be a plausible position.
That said, I think it depends how you use it. You can learn from explanations, and you'd better avoid "rewrite this for me and do nothing else" kind of approach.
> There are a lot of people who go through life by vibing. And honestly: that’s not automatically “bad.” Sometimes it’s even the only workable way to get through things. The issue is that “vibe-first” people tend to have a pretty loose relationship with truth, rigor, and being pinned down by specifics. They’ll confidently move forward on what sounds right instead of what they can verify.
I'll finish this post with a sentence containing an em-dash -- just to confuse people -- and by remarking on how sad I find it that people latch onto dashes and complete sentences as the signifiers of LLM use, instead of the inconsistent logic and general sloppiness that's the actual problem.
I'm not saying that I'm no longer dealing with code at all though. The way I work is interactively with the LLM and pretty much tell it exactly what to do and how to do it. Sometimes all the way down to "don't copy the reference like that, grab a deep copy of the object instead". Just like with any other type of programming, the only way to achieve valuable and correct results is by knowing exactly what you want and express that exactly and without ambiguity.
But I no longer need to remember most of the syntax for the language I happen to work with at the moment, and can instead spend time thinking about the high level architecture. To make sure each involved component does one thing and one thing well, with its complexities hidden behind clear interfaces.
Engineers who refuse to, or can't, or won't utilize the benefits that LLMs bring will be left behind. It's just the way it is. I'm already seeing it happening.
But it absolutely has to be combined with verification/testing at the same speed as code production.
reminds me of the experience of reading a math text without doing the exercises, thinking that you've understood the material, and then falling flat on your face when you attempt to apply your "understanding" to a novel problem. there's a significant difference between passively reading something and really putting active effort into it. only the latter leads to actual understanding ime
On the other hand, LLM-generated code comments better than I do, so given a long enough time horizon, it could be more understandable at a later time than code I've written myself (we've all had the experience of forgetting how things work).
> On the other hand, LLM-generated code comments better than I do, so given a long enough time horizon, it could be more understandable at a later time than code I've written myself (we've all had the experience of forgetting how things work).
[1] https://en.wikipedia.org/wiki/Spaced_repetition
You ask questions about code when you implement something and if you cannot answer these questions, you go to code to find answers out and refresh your understanding of it.
For this to work you have to be interested in the understanding of the code and code should be created at the pace you can keep up.
Software engineers usually do create code economically because they need to remember and understand it. Vibe coders do not have this particular constraint, they just do not aim for most understandable code possible. Even if there are more comments in code.
However, this comes at the cost of losing track of the minute details of the implementation because you didn't write it yourself. I find it a bit analogous to code I've reviewed vs code I've written.
However, I've found using AI for code structure summary and questioning tends to be a good way to get around it. I might forget faster, but I also pick it up faster.
Then, after it says, yes I'm sure this is production ready and we're good to move on, you have Codex and Gemini both review it one last time, and ask it to address their feedback if it's valuable or not.
After all this, it's the only time I'll look at the code and review it and make sure it's coherent.
Until then, I assume it's garbage.
I'd estimate this still improves velocity by 10x, and more importantly, allows me to operate at a pace I couldn't without burning out.
You're just getting less work done on a slower cadence and asking the questions in design review and in code reviews...
i don't mind managing people, but i don't want to manage machines unless i can control them with the precise languages that the commandline and programming languages use. prompting a LLM is to vague an interface for me, the outcome is to unreliable, to unpredictable.
Any examples how you see some engineers being left behind?
I don't know where you live, but around where I live in Denmark you'd fail for not using AI at a senior interview in a lot of places. Even places which aren't exactly AI fans use AI to some extend.
The biggest challenge we face right now is figuring out how you create developers who have enough experience to know how to use the AI tools in a critical manner. Especially because you're typically given agents for various taks, which are already configured to know how we want things to be written.
Additionally there are the AI targets set by C suites based on what everyone is saying on TV, and what we can actually deliver based on the available data sets, integration points, and naturally those sign offs for data governance, and hallucinations guardrails.
If you can’t interview without immediately reaching for an LLM you are considered unfit to work here.
> If you don't jump off the cliff you're falling behind
It is anecdotal for sure, but it's a pattern that seems to be emerging around me that expectations of velocity increases, and those who don't use AI can't keep up.
Kind of.... I don't know. To get placed such requirements from the top down and not fight back, just take it head on, not even maliciously, don't even oppose it on a technical basis, just be like "yeah, you've now gotta ship faster or you're left behind, so therefore LLMs must be the future!", no critical thought attached. Is this shit coming from experienced engineers?
Preposterous we're relying on "it's better because I feel like", "dudes who don't use it are falling behind at work", "they ask for it in job interviews".
It would be an honor to be “left behind” by people who practice their craft with such carelessness.
(Frankly, I should probably stop replying to self-professed LLM boosters entirely since there’s a good chance I’m just chatting with an LLM.)
FWIW, I've opened a half dozen PRs from LLMs and had them approved. I have some prompts I use to make them very difficult to tell they are AI.
However if it is a big anti-llm project I just fork and have agents rebase my changes.
> utterly stupid
That's completely uncalled for.
EDIT: What exactly do you mean by:
> most likely reviewed.
Let's say every line was actually reviewed. That's still nowhere near good enough. The changes are being reviewed by the wrong people. Not the maintainers of the project, just some random folks who have inherited a vibecoded fork.