I am constantly surprised how prevalent this attitude is. ChatGPT was only just released in 2022. Is there some expectation that these things won't improve?
> LLM’s never provide code that pass my sniff test
This is ego speaking.
I definitely expect them to improve. But I also think the point at which they can actually replace a senior programmer is pretty much the exact point at which they can replace any knowledge worker, at which point western society (possibly all society) is in way deeper shit than just me being out of a job.
> This is ego speaking.
It definitely isn't. LLMs are useful for coding now, but they can't really do the whole job without help - at least not for anything non-trivial.
As another senior developer I won't say it's impossible that I'll ever benefit from code generation but I just think it's a terrible space to try and build a solution - we don't need a solution here - I can already type faster than I can think.
I am keenly interested in seeing if someone can leverage AI for query performance tuning or, within the RDBMS, query planning. That feels like an excellent (if highly specific) domain for an LLM.
Pay the $20 for Claude, copy the table DDL's in along with a query you'd like to tune.
Copy in any similar tuned queries you have and tell it you'd like to tune your query in a similar manner.
Once you've explained what you'd like it to do and provided context hit enter.
I'd be very surprised if having done this you can't find value in what it generates.
But can you write tickets faster than you can implement them? I certainly can.
Personally, I have a tendency at work to delay creating tickets until after I've already written the implementation.
Why? Because tickets in my employer's system are expected to identify which component needs to be changed, and ideally should have some detail about what needs to be changed. But both of those things depend on the design of the change being implemented.
In my experience, any given feature usually has multiple possible designs, and the only way to know if a design is good is to try implementing it and see how clean or messy it ends up. Of course, I can guess in advance which design will turn out well. I have to guess, or else I wouldn't know which design to try implementing first. But often my first attempt runs into unexpected wrinkles and I retreat and try a different design.
Other people will start with a design and brute-force their way to working code, and (in my judgmental opinion) the code often ends up lower-quality because of it.
Sooner or later, perhaps AI will be able to perform that entire process autonomously, better than I can. In the meantime, though, people often talk about using AI like a 'junior engineer', where you think up a design yourself and then delegate the grunt work to the AI. That approach feels flawed to me, because it disconnects the designer from the implementer.
abso-frigging-lutely
To me this is n example of software being a form Of literacy - creative work. And yet process is designed by software illiterates who think novels can be written by pre-planning all the paragraphs
If it's "Get us to the moon", it's gonna take me years to write that ticket.
If it was "Make the CTA on the homepage red", it is up for debate whether I needed a ticket at all.
That would be great if the reality wasn't that the suggested LLM slop is actually making it harder to get the much better intellisense suggestions of last year.
When hand-held power tools became a thing, the Hollywood set builder’s union was afraid of this exact same thing - people would be replaced by the tools.
Instead, productions built bigger sets (the ceiling was raised) and smaller productions could get in on things (the floor was lowered).
I always took that to mean “people aren’t going to spend less to do the job - they’ll just do a bigger job.”
A few hundred animators turned into a few thousand computer animators & their new support crew, in most shops. And new, smaller shops took form! But the shops didn't go away, at least not the ones who changed.
It basically boils down to this: some shops will act with haste and purge their experts in order to replace them with LLMs, and others will adopt the LLMs, bring on the new support staff they need, and find a way to synthesize a new process that involves experts and LLMs.
Shops who've abandoned their experts will immediately begin to stagnate and produce more and more mediocre slop (we're seeing it already!) and the shops who metamorphose into the new model you're speculating at will, meanwhile, create a whole new era of process and production. Right now, you really want to be in that second camp - the synthesizers. Eventually the incumbents will have no choice but to buy up those new players in order to coup their process.
Then the steam engine and internal combustion engine came around and work horses all but disappeared.
There's no economic law that says a new productivity-enhancing programming tool is always a stirrup and never a steam engine.
All the tools that are stirrups were used "by the horse" (you get what I mean); that implies to me that so long as the AI tools are used by the programmers (what we've currently got), they're stirrups.
The steam engines were used by the people "employing the horse" - ala, "people don't buy drills they buy holes" (people don't employ horses, they move stuff) - so that's what to look for to see what's a steam engine.
IMHO, as long as all this is "telling the computer what to do", it's stirrups, because that's what we've been doing. If it becomes something else, then maybe it's a steam engine.
And, to repeat - thank you for this point, it's an excellent one, and provides some good language for talking about it.
Maybe another interesting case would be secretaries. It used to be very common that even middle management positions at small to medium companies would have personal human secretaries and assistants, but now they're very rare. Maybe some senior executives at large corporations and government agencies still have them, but I have never met one in North America who does.
Below that level it's become the standard that people do their own typing, manage their own appointments and answer their own emails. I think that's mainly because computers made it easy and automated enough that it doesn't take a full time staffer, and computer literacy got widespread enough that anyone could do it themselves without specialized skills.
So if programming got easy enough that you don't need programmers to do the work, then perhaps we could see the profession hollow out. Alternatively we could run out of demand for software but that seems less likely!
(a related article: https://archive.is/cAKmu )
Those textile workers were afraid machines would replace them, but that didn't happen - the work was sent overseas, to countries with cheaper labor. It was completely tucked away from regulation and domestic scrutiny, and so remains to this day a hotbed of human rights abuses.
The phenomenon you're describing wasn't an industry vanishing due to automation. You're describing a moment where a domestic industry vanished because the cost of overhauling the machinery in domestic production facilities was near to the cost of establishing an entirely new production facility in a cheaper, more easily exploitable location.
[0] https://www.uniformmarket.com/statistics/global-apparel-indu...
Computers aren't going anywhere, so the whole field of programming will continue to grow, but will there still be FAANG salaries to be had?
if you're afraid of salaries shrinking due to LLMs, then i implore you, get out of software development. it'll help me a lot!
Software is typically not a cost constrained activity due to its typically higher ROI/scale. Its all about fixed costs and scaling profits mostly. Unfortunately given this my current belief is that on balance AI will destroy many jobs in this industry if it gets to the point where it can do a software job.
Assuming inelastic demand (software demand relative to SWE costs) any cost reductions in inputs (e.g. AI) won't translate to much more demand in software. The same effect that drove SWE prices high and didn't change demand for software all that much (explains the 2010's IMO particularly in places like SV) also works in reverse.
- Software scales; generally it is a function of market size, reach, network effects, etc. Cost is a factor but not the greatest factor. Most software makes profit "at scale" - engineering is a capital fixed cost. This means that software feasibility is generally inelastic to cost or rather - if I make SWE's cheaper/not required to employ it wouldn't change the potential profit vs cost equation much IMV for many different opportunities. Software is limited more by ideas, and potential untapped market opportunities. Yes - it would be much cheaper to build things; but it wouldn't change the feasibility of a lot of project assessments since cost of SWE's at least from what I've seen in assessments isn't the biggest factor. This effect plays out to varying degrees in a lot of capex - as long as the ROI makes sense its worth going ahead especially for larger orgs who have more access to capital. The ROI from scale dwarfs potential cost rises often making it less of a function of end SWE demand. This effect happens in other engineering disciplines as well to varying effects - software just has it in spades until you mix hardware scaling into the mix (e.g GPUs).
- The previous "golden era" where the in-elasticity of software demand w.r.t cost meant salaries just kept rising. Inelasticity can be good for sellers of a commodity if demand increases. More importantly demand didn't really decrease for most companies as SWE salaries kept rising - entry requirements were relaxing generally. The good side of in-elasticity is reversed potentially by AI making it a bad thing.
However small "throwaway" software which does have a "is it worth it/cost factor" will increase under AI most probably. Just don't think it will offset the reduction demanded by capital holders; nor will it be necessarily done by the same people anyway (democratizing software dev) meaning it isn't a job saver.
In your case I would imagine there is a reason why said software doesn't have SWE's coding it now - it isn't feasible given the likely scale it would have (I assume just your team). AI may make it feasible, but that doesn't help the OP in the way it does it - it does so by making it feasible for as you put it junior out of uni not even "SWE" grads. That doesn't help the OP.
Also, a piece missing from this comparison is a set of people who don't believe the new tool will actually have a measurable impact on their domain. I assume few-to-none could argue that power tools would have no impact on their profession.
The history of software production as a profession (as against computer science) is essentially a series of incremental increases in the size and complexity of systems (and teams) that don't fall apart under their own weight. There isn't much evidence we have approached the limit here, so it's a pretty good bet for at least the medium term.
But focusing on system size is perhaps a red herring. There is an almost unfathomably vast pool of potential software systems (or customization of systems) that aren't realized today because they aren't cost effective...
The demand for software improvements is effectively inexhaustible. It’s not a zero sum game.
Have people seen some of the recent software being churned out? Hint, it's not all GenAI bubblespit. A lot of it is killer, legitimately good stuff.
you know what i'd do if AI made it so i could replace 10 devs with 8? use the 2 newly-freed up developers to work on some of the other 100000 things i need done
It's not about a discrete product or project, but continuous improvement upon that which already exists is what makes up most of the volume of "what would happen if we had more people".
- a good LIMS (Laboratory Information Management System) that incorporates bioinformatics results. LIMS come from a pure lab, benchwork background, and rarely support the inclusion of bioinformatics analyses on samples included in the system. I have yet to see a lab that uses an off-the-shelf LIMS unmodified - they never do what they say they do. (And the amount of labs running on something built on age-old software still in use is... horrific. I know one US lab running some abomination built on Filemaker Pro)
- Software to manage grants. Who is being owed what, what are the milestones, who's looking after this, who are the contact persons, what are the milestones and when to remind, due diligence on potential partners etc. I worked for a grant-giving body and they came up with a weird mix of PowerBI and a pile of Excel sheets and PDFs.
- A thing that lets you catalogue Jupyter notebooks and Rstudio projects. I'm drowning in various projects from various data scientists and there's no nice way to centrally catalogue all those file lumps - 'there was this one function in this one project.... let's grep a bit' can be replaced by a central findable, searchable, taggable repository of data science projects.
Oh... oh my. This extends so far beyond data science for me and I am aching for this. I work in this weird intersection of agriculture/high-performance imaging/ML/aerospace. Among my colleagues we've got this huge volume of Excel sheets, Jupyter notebooks, random Python scripts and C++ micro-tools, and more I'm sure. The ones that "officially" became part of a project were assigned document numbers and archived appropriately (although they're still hard to find). The ones that were one-off analyses for all kinds of things are scattered among OneDrive folders, Zip files in OneDrive folders, random Git repos, and some, I'm sure, only exist on certain peoples' laptops.
Well then you can count IDEs, static typing, debuggers, version control etc. as replacing programmers too. But I don't think any of those performance enhancers have really reduced the number of programmers needed.
In fact it's a well known paradox that making a job more efficient can increase the number of people doing that job. It's called the Jevons paradox (thanks ChatGPT - probably wouldn't have been able to find that with Google!)
Making people 20% more efficient is very different to entirely replacing them.
Agree with this take. I think the probability that this happens within my next 20 years of work is very low, but are non-zero. I do cultivate skills that are a hedge against this, and if time moves on and the probability of this scenario seems to get larger and larger, I'll work harder on those skills. Things like fixing cars, welding, fabrication, growing potatoes, etc (which I already enjoy as hobbies). As you said, skills that are helpful if shit were to really hit the fan.
I think there are other "knowledge workers" that will get replaced before that point though, and society will go through some sort of upheaval as this happens. My guess is that capital will get even more consolidated, which is sort of unpleasant to think about.
earnest question because i still consider this a vague, distant future: how did you come up with 20 years?
Its the mid age people 35-45 yr olds that I think will be hit hardest by this if it eventuates. Usually at this point in life there's plenty of commitments (family, mortgage, whatever). The career may end before they are able to retire, but they are either too burdened with things; or ageism sets in making it hard to be adaptable.
And like the parent poster said - even if you were to avoid the industry, what would you pivot to? Anything else you might go into would be trivial to automate by that point.
Those industries also more wisely IMO tend to be unionised, own their own business, and so have the incentive to keep their knowledge tight. Even with automation the customer (their business) and the supplier (them and their staff) are the same so the value transfer of AI will make their job easier but they will keep the value. All good things to slow down progress and keep some economic rent for yourself and your family. Slow change is a less stressful life.
The intellectual fields lose in an AI world long term; the strong and the renter class (capital/owners) win. That's what "Intelligence is a commodity" that many AI heads keep saying actually means. This opens up a lot of future dystopian views/risks that probably aren't worth the benefit IMO to the majority of people that aren't in the above classes of people (i.e. most people).
The problem with software in general is that it is quite difficult generally to be a "long term" founder at least in my opinion for most people which means most employment comes from large corps/govt's/etc where the supplier (the SWE) is different than the employer. Most ideas generally don't make it or last only briefly, and the ones that stick around usually benefit from dealing with scale - something generally only larger corps have with their ideas (there are exceptions in new fields, but then they become the next large corp and there isn't enough for everyone to do this).
When this moment becomes reality - the world economy will change a lot, all jobs and markets will shift.
And there won’t be any way to future proof your skills and they all will be irrelevant.
Right now many like to say “learn how to work with ai, it will be valuable “ . No, it won’t. Because even now it is absolutely easy to work with it, any developer can pick up ai in a week, and it will become easier and easier.
A better time spent is developing evergreen skills.
*sort of, sometimes, with simple enough problems with sufficiently little context, for code that can be easily tested, and for which sufficient examples exist in the training data.
I mean hey, two years after being promised AGI was literally here, LLMs are almost as useful as traditional static analysis tools!
I guess you could have them generate comments for you based on the code as long as you're happy to proofread and correct them when they're wrong.
Remember when CPUs were obsolete after three years? GPT has shown zero improvement in its ability to generate novel content since it was first released as GPT2 almost ten years ago! I would know because I spent countless hours playing with that model.
Secondly, LLMs are objectively useful for coding now. That's not the same thing as saying they are replacements for SWEs. They're a tool, like syntax highlighting or real-time compiler error visibility or even context-aware keyword autocompletion.
Some individuals don't find those things useful, and prefer to develop in a plain text editor that does not have those features, and that's fine.
But all of those features, and LLMs are now on that list, are broadly useful in the sense that they generally improve productivity across the industry. They already right now save enormous amounts of developer time, and to ignore that because you are not one of the people whose time is currently being saved, indicates that you may not be keeping up with understanding the technology of your field.
There's an important difference between a tool being useful for generating novel content, and a tool being useful. I can think of a lot of useful tools that are not useful for generating novel content.
But is that actually a true statement. Are there actual studies to back that up?
AI is hyped to the moon right now. It is really difficult to separate the hype from reality. There are ancedotal reports of ai helping with coding, but there are also ancedotal reports that they get things almost right but not quite, which often leads to bugs which wouldn't otherwise happen. I think its unclear if that is a net win for productivity in software engineering. It would be interesting if there was a robust study about it.
I am aware of an equal number of studies about the time saved overall by use of LLMs, and time saved overall by use of syntax highlighting.
In fact, here's a study claiming syntax highlighting in IDEs does not help code comprehension: https://link.springer.com/article/10.1007/s10664-017-9579-0
Shall we therefore conclude that syntax highlighting is not useful, that developers who use syntax highlighting are just part of the IDE hype train, and that anecdotal reports of syntax highlighting being helpful are counterbalanced by anecdotal reports of $IDE having incorrect syntax highlighitng on $Esoteric_file_format?
Most of the failures of LLMs with coding that I have seen has been a result of asking too much of the LLM. Writing a hundred context-aware unit tests is something that an LLM is excellent at, and would have taken a developer a long time previously. Asking an LLM to write a novel algorithm to speed up image processing of the output of your electron microscope will go less well.
Yes. We should conclude that syntax highlighting is not useful in languages that the syntax highlighter does not support. I think basically everyone would agree with this statement.
Similarly an llm that worked 100% of the time and could solve any problem would be pretty useful. (Or at least worked correctly as often as syntax highlighting in situations where it is actually used does)
However that's not the world we live in. Its a reasonable question to ask if llm is good enough yet where the productivity gain outweighs the productivity lost.
The point I was trying to make was, an LLM is as reliably useful as syntax highlighting, for the tasks that coding LLMs are good at today. Which is not a lot, but enough to speed up junior devs. The issues come when people assume they can solve any problem, and try to use them on tasks to which they are not suited. Much like applying syntax highlighting on an unsupported language, this doesn't work.
Like any tool, there's a learning curve. Once someone learns what does and does not work, it's generally an strict productivity boost.
I fixed a production issue earlier this year that turned out to be a naive infinite loop - it was trying to load all data from a paginated API endpoint, but there was no logic to update the page number being fetched.
There was a test for it. Alas, the test didn't actually cover this scenario.
I mention this because it was committed by a co-worker whose work is historically excellent, but who started using Copilot / ChatGPT. I'm pretty sure it was an LLM-generated function and test, and they were deeply broken.
Mostly they've been working great for this co-worker.
But not reliably.
A very similar example is StackOverflow. If you copy/paste answers verbatim from SO, you will have problems. Some top answers are deeply broken or have obvious bugs. Frequently, SO answers are only related to your question, but do not explicitly answer it.
SO is useful to the industry in the same way LLMs are.
LLMs are in the middle. Its unclear which side of the line they are on. Some ancedotes say one thing some say another. That's why studies would be great. Its also why syntax highlighting is a bad comparison since that is not in the greyzone.
No, they're subjectively useful for coding in the view of some people. I find them useless for coding. If they were objectively useful, it would be impossible for me to find them useless because that is what objectivity means.
It's useless to your coding, but useful to the industry of coding.
If that statement isn't coming from ego, then where is it coming from? It's provably true that LLM's can generate working code. They've been trained on billions of examples.
Developers seem to focus on the set of cases that LLM's produce code that doesn't work, and use that as evidence that these tools are "useless".
It has been interesting as a rubber duck, exploring a new topic or language, some code golf, but so far not for production code for me.
Now, understand that most people don't have the same grasp of [your programming language] that you have, so it's probably not easier for them to write it.
I actually said in my comment that exploring a new language is one area I find LLMs to be interesting.
They produce mostly working code, often with odd design decisions that the bot can't fully justify.
The difficult part of coding is cultivating a mental model of the problem + solution space. When you let an LLM write code for you, your mental model falls behind. You can read the new code closely, internalize it, double-check the docs, and keep your mental model up to date (which takes longer than you think), or you can forge ahead, confident that it all more or less looks right. The second option is easier, faster, and very tempting, and it is the reason why various studies have found that code written with LLM assistance introduces more bugs than code written without.
There are plenty of innovations that have made programming a little bit faster (autocomplete, garbage collection, what-have-you), but none of them were a silver bullet. LLMs aren't either. The essential complexity of the work hasn't changed, and a chat bot can't manage it for you. In my (admittedly limited) experience with code assistants, I've found that the faster you move with an LLM, the more time you have to spend debugging afterwards, and the more difficult that process becomes.
if the stakeholders knew how to do what they needed to build and how, then they could use LLMs, but translating complex requirements into code is something that these tools are not even close to cracking.
Completely agree.
What I don't agree with is statements like these:
> LLM’s never provide code that pass my sniff test
To me, these (false) absolutions about chat bot capabilities, are being rehashed so frequently, that it derails every conversation about using LLM's for dev work. You'll find similar statements in nearly every thread about LLM's for coding tasks.
It's provably true that LLM's can produce working code. It's also true, that some increasingly large portion of coding is being offloaded to LLM's.
In my opinion, developers need to grow out of this attitude that they are John Henry and they'll outpace the mechanical drilling machine. It's a tired conversation.
You've restated this point several times but the reason it's not more convincing to many people is that simply producing code that works is rarely an actual goal on many projects. On larger projects it's much more about producing code that is consistent with the rest of the project, and is easily extensible, and is readable for your teammates, and is easy to debug when something goes wrong, is testable, and so on.
The code working is a necessary condition, but is insufficient to tell if it's a valuable contribution.
LLM's can sometimes provide the bare minimum. And then you have to refactor and massage it all the way to the good bit, but unlike looking up other people's endeavors on something like Stack Overflow, with the LLM's code I have no context why it "thought" that was a good idea. If I ask it, it may parrot something from the relevant training set, or it might be bullshitting completely. The end result? This is _more_ work for a senior dev, not less.
Hence why it has never passed my sniff test. Its code is at best the quality of code even junior developers wouldn't open a PR for yet. Or if they did they'd be asked to explain how and why and quickly learn to not open the code for review before they've properly considered the implications.
This is correct - but it's also true that LLMs can produce flawed code. To me the cost of telling whether code is correct or flawed is larger than the cost of me just writing correct code. This may be an AuDHD thing but I can better comprehend the correctness of a solution if I'm watching (and doing) the making of that solution than if I'm reading it after the fact.
From what I've seen of Copilots, while they can produce working code, I've not seen that much that it offers beyond the surface level which is fast enough for me to type. I am also deeply perturbed from some interviews I've done for senior candidates recently who are using them and, when asked to disable them for collaborative coding task, completely fall apart because of their dependency over knowledge.
This is not to say I do not see value in AI, LLMs or ML (I very much do). However, I code broadly at the speed of thought, and that's not really something I think will be massively aided by it.
At the same time, I know I am an outlier in my practice relative to lots around me.
While I don't doubt other improvements that may come from LLM in development, the current state of the art feels less like a mechanical drill and more like an electric triangle.
Senior devs know this, and factor code down to the minimum necessary.
Junior devs and LLMs think that writing code is the point and will generate lots of it without worrying about things like leverage, levels of abstraction, future extensibility, etc.
You can write good code or bad code with LLMs.
Of course if you try to one shot something complex with a single line prompt, the result will be bad. This is why humans are still needed and will be for a long time imo.
Empirically, LLMs work best at coding when doing completely "routine" coding tasks: CRUD apps, React components, etc. Because there's lots of examples of that online.
I'm writing a data-driven query compiler and LLM code assistance fails hard, in both blatant and subtle ways. There just isn't enough training data.
Another argument: if a LLM could function like a senior dev, it could learn to program in new programming languages given the language's syntax, docs and API. In practice they cannot. It doesn't matter what you put into the context, LLMs just seem incapable of writing in niche languages.
Which to me says that, at least for now, their capabilities are based more on pattern identification and repetition than they are on reasoning.
That said, you’re right of course that it will do better when there’s more training data.
ChatGPT, even now in late-2024, still hallucinates standard-library types and methods more-often-than-not whenever I ask it to generate code for me. Granted, I don’t target the most popular platforms (i.e. React/Node/etc; I’m currently in a .NET shop, which is a minority platform now, but ChatGPT’s poor performance is surprising given the overall volume and quality of .NET content and documentation out there.
My perception is that “applications” work is more likely to be automated-away by LLMs/copilots because so much of it is so similar to everyone else’s, so I agree with those who say LLMs are only as good as there are examples of something online, whereas asking ChatGPT to write something for a less-trodden area, like Haskell or even a Windows driver, is frequently a complete waste of time as whatever it generates is far beyond salvaging.
Beyond hallucinations, my other problem lies in the small context window which means I can’t simply provide all the content it needs for context. Once a project grows past hundreds of KB of significant source I honestly don’t know how us humans are meant to get LLMs to work on them. Please educate me.
I’ll declare I have no first-hand experience with GitHub Copilot and other systems because of the poor experiences I had with ChatGPT. As you’re seemingly saying that this is a solved problem now, can you please provide some details on the projects where LLMs worked well for you? (Such as which model/service, project platform/language, the kinds of prompts, etc?). If not, then I’ll remain skeptical.
Not an argument, unsolicited advice: my guess is you are asking it to do too much work at once. Make much smaller changes. Try to ask for as roughly much as you would put into one git commit (per best practices)-- for me that's usually editing a dozen or less lines of code.
> Once a project grows past hundreds of KB of significant source I honestly don’t know how us humans are meant to get LLMs to work on them. Please educate me.
https://github.com/Aider-AI/aider
Edit: The author of aider puts the percentage of the code written by LLMs for each release. It's been 70%+. But some problems are still easier to handle yourself. https://github.com/Aider-AI/aider/releases
Then why can't I see this magical code that is produced? I mean a real big application with a purpose and multiple dependencies, not yet another ReactJS todo list. I've seen comments like that a hundred times already but not one repository that could be equivalent to what I currently do.
For me the experience of LLM is a bad tool that calls functions that are obsolete or do not exist at all, not very earth-shattering.
They don't have to replace you to reduce headcount. They could increase your workload so where they needed five senior developers, they can do with maybe three. That's like six one way and half a dozen the other way because two developers lost a job, right?
What I've seen of them, the good ones mostly produce OK code. Not terrible, usually works.
Although I like them even for that low-ish bar, although I find them to be both a time-saver and a personal motivation assistant, they're still a thing that needs a real domain expert to spot the mistakes they make.
> Developers seem to focus on the set of cases that LLM's produce code that doesn't work, and use that as evidence that these tools are "useless".
I do find it amusing how many humans turned out to be stuck thinking in boolean terms, dismissing the I in AGI, calling them as "useless" because it "can't take my job". Same with the G in AGI, dismissing the breadth of something that speaks 50 languages when humans who speak five or six languages are considered unusually skilled.
I am pro AI and I'm probably even overvaluing the value AI brings. However, for me, this doesn't work in more "esoteric" programming languages or those with stricter rulesets like Rust. LLMS provide fine JS code, since there's no compiler to satisfy, but CPP without undefined behaviour or compiling Rust code is rare.
There's also no chance of LLMS providing compiling code if you're using a library version with a newer API than the one in the training set.
Even if it did work, working code is barely half the battle.
Yeah for simple examples, especially in web dev. As soon as you step outside those bounds they make mistakes all the time.
As I said, they're still useful because roughly correct but buggy code is often quite helpful when you're programming. But there's zero chance you can just say "write me an driver for the nRF905 using Embassy and embedded-hal" and get something working. Whereas I, a human, can do that.
https://chatgpt.com/share/6760c3b3-bae8-8009-8744-c25d5602bf...
Because, one way or another, you're still going to need to become fluent enough in the problem domain & the API that you can fully review the implementation and make sure chatgpt hasn't hallucinated in any weird problems. And chatgpt can't really explain its own work, so if anything seems funny, you're going to have to sleuth it out yourself.
And at that point, it's just 339 lines of code, including imports, comments, and misc formatting. How much time have you really saved?
I think that software development is just an extremely poor market segment for these kinds of tools - we've already got mountains of productivity tools that minimize how much time we need to spend doing the silly rote programming stuff - most of software development is problem solving.
Given the growth-oriented capitalist society we live in in the west, I'm not all that worried about senior and super-senior engineers being fired. I think a much more likely outcome is that if a business does figure out a good way to turn an LLM into a force-multiplier for senior engineers, they're going to use that to grow faster.
There is a large untapped niche too that this could potentially unlock: projects that aren't currently economically viable due to the current cost of development. I've done a few of these on a volunteer basis for non-profits but can't do it all the time due to time/financial constraints. If LLM tech actually makes me 5x more productive on simple stuff (most of these projects are simple) then it could get viable to start knocking those out more often.
No, it really isn't. Repeatedly, the case is that people are trying to pass off GPT's work as good without actually verifying the output. I keep seeing "look at this wonderful script GPT made for me to do X", and it does not pass code review, and is generally extremely low quality.
In one example, a bash script was generated to count number SLoC changed by author; it was extremely convoluted, and after I simplified it, I noticed that the output of the simplified version differed, because the original was omitted changes that were only a single line.
In another example it took several back & forths during a review to ask "where are you getting this code? / why do you think this code works, when nothing in the docs supports that?" and after several back and forths, it was admitted that GPT wrote it. The dev who wrote it would have been far better served RTFM, than a several cycle long review that ended up with most of GPT's hallucinations being stripped from the PR.
Those who think LLM's output is good have not reviewed the output strenuously enough.
> Is there some expectation that these things won't improve?
Because randomized token generation inherently lacks actual reasoning about the behavior of the code. My code generator does not.
The moment you have some weird library that 4 people in the world know (which happens more than you’d expect) or hell even something without a lot of OSS code what exactly is an LLM going to do? How is it supposed to predict code that’s not derived from its training set?
My experience thus far is that it starts hallucinating and it’s not really gotten any better at it.
I’ll continue using it to generate sed and awk commands, but I’ve yet to find a way to make my life easier with the “hard bits” I want help with.
The first example I gave was an example of someone using an LLM to generate sed & awk commands, on which it failed spectacularly, on everything from the basics to higher-level stuff. The emitted code even included awk, and the awk was poor quality: e.g., it had to store the git log output & make several passes over it with awk, when in reality, you could just `git log | awk`; it was doing `... | grep | awk` which … if you know awk, really isn't required. The regex it was using to work with the git log output it was parsing with awk was wrong, resulting in the wrong output. Even trivial "sane bash"-isms, it messed up: didn't quote variables that needed to be quotes, didn't take advantage of bashisms even though requiring bash in the shebang, etc.
The task was a simple one, bordering on trivial, and any way you cut it, from "was the code correct?" to "was the code high quality?", it failed.
But it shouldn't be terribly surprising that an LLM would fail at writing decent bash: its input corpus would resemble bash found on the Internet, and IME, most bash out there fails to follow best-practice; the skill level of the authors probably follows a Pareto distribution due to the time & effort required to learn anything. GIGO, but with way more steps involved.
I've other examples, such as involving Kubernetes: Kubernetes is also not in the category of "4 people in the world know": "how do I get the replica number from a pod in a statefulset?" (i.e., the -0, -1, etc., at the end of the pod name) — I was told to query,
.metadata.labels.replicaset-序号
It really is. Either that or you’re not thinking about what you’re saying.
Imagine code passes your rigorous review.
How do you know that it wasn’t from an LLM?
If it’s because you know that you only let good code pass your review and you know that LLMs only generate bad code, think about that a bit.
That's not what I'm saying (and it's a strawman; yes, presumably some LLM code would escape review and I wouldn't know it's from an LLM, though I find that unlikely, given…) — what I'm saying is of LLM generated code that is reviewed, what is the quality & correctness of the reviewed code? And it's resoundingly (easily >90%) crap.
Obviously we can't sample from unknown-authorship … nor am I; I'm sampling problems that I and others run through an LLM, and the output thereof.
The other facet of this point is that I believe a lot of the craze that users using the LLM have is driven by them not looking closely at the output; if you're just deriving code from the LLM, chucking it over the wall, and calling it a day (as was the case from one of the examples in the comment above) — you're perceiving the LLM as being useful, when it fact it is leaving bugs that you're either not attributing to it, someone else is cleaning up (again, that was the case in the above example), etc.
What makes you so sure that none of the resoundingly non-crap that you have reviewed was not produced by LLM?
It’s like saying you only like homemade cookies not ones from the store. But you may be gleefully chowing down on cookies that you believe are homemade because you like them (so they must be homemade) without knowing they actually came from the store.
From the post you're replying to:
> Obviously we can't sample from unknown-authorship … nor am I; I'm sampling problems that I and others run through an LLM, and the output thereof.
> I'm sampling problems that I and others run through an LLM
This is not what’s happening unless 100% of the problems they’ve sampled (even outside of this fun little exercise) have been run through an LLM.
They’re pretending like it doesn’t matter that they’re looking at untold numbers of other problems and are not aware whether those are LLM generated or not.
No. The accurate way to say this is:
“The code that they have reviewed that they know came from an LLM has been, to their standards, subpar.”
A human junior developer can learn from this tutoring and rarely regress over time. But the LLM’s all by design cannot and do not rewire their understanding of the problem space over time, nor do they remember examples and lessons from previous iterations to build upon. I have to handhold them forever, and they never learn.
Even when they use significant parts of the existing codebase as their context window they’re still blind to the whole reality and history of the code.
Now just to be clear, I do use LLM’s at my job. Just not to code. I use them to parse documents and assist users with otherwise repetitive manual tasks. I use their strength as language models to convert visual tokens parsed by an OCR to grasp the sentence structure and convert that into text segments which can be used more readily by users. At that they are incredible, even something smaller like llama 7b.
Is there any expectations that things will? Is there more untapped great quality data that LLMs can ingest? Will a larger model perform meaningfully better? Will it solve the pervasive issue of generating plausibly sounding bullshit?
I used LLMs for a while, I found them largely useless for my job. They were helpful for things I don't really need help with, and they were mostly damaging for things I actually needed.
> This is ego speaking.
Or maybe it was an accurate assessment for his use case, and your wishful thinking makes you think it was his ego speaking.
Seems like an odd question. The answer is obviously yes: There is a very pervasive expectation that LLM's will continue to improve, and it seems odd to suggest otherwise. There is hundreds of billions of dollars being spent on AI training and that number is increasing each year.
> Is there more untapped great quality data that LLMs can ingest?
Why wouldn't there be? AI's are currently trained on the internet but that's obviously not the only source of data.
> Will a larger model perform meaningfully better?
The answer to this, is also yes. It is well established that, all else being equal, a bigger model is better than a smaller model, assuming that the smaller model hasn't already captured all of the available information.
It isn't odd at all. In the early 21st century there were expectations of ever, exponentially increasing processing power. This misconception partially gave us the game Crysis, which, if I'm not mistaken, was written with wildly optimistic assumptions about the growing power of computer hardware.
People are horrible at predicting the future of technology (beyond meaninglesslessly or trivially broad generalizations) and even when predictions turn out to be correct, they're often correct for the wrong reasons. If we were better at it, even in the shortest term, where such predictions should be the easiest, we'd all be megamillionaires, because we'd have seen the writing on the wall and invested in Nvidia before the AI craze reached its current fever pitch.
I did this when I saw the first stable diffusion AI titties. So far up over 10x.
If I had a nickel for every tech that takes off once porn finds its way to it, I wouldn't be counting in nickels.
YMMV but count me in the camp that I think there’s better odds that LLMs are at or near their potential vs in their nascent stages.
That makes an assumption that throwing dollars on AI training is a surefire way to solve the many shortcomings of LLMs. It is a very optimistic assumption.
> Why wouldn't there be? AI's are currently trained on the internet but that's obviously not the only source of data.
"The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.
> The answer to this, is also yes. It is well established that, all else being equal, a bigger model is better than a smaller model, assuming that the smaller model hasn't already captured all of the available information.
I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.
The assumption that generative AI will improve is as valid as the assumption that it will plateau. It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.
That's not an assumption that I am personally making. That's what experts in the field believe.
> "The Internet" basically encompasses all meaningful sources of data available, especially if we are talking specifically about software development. But even beyond that, it is very unclear what other high quality data it would consume that would improve the things.
How about, interacting with the world?
> I love how you conveniently sidestepped the part where I ask if it would improve the pervasive issue of generating plausibly sounding bullshit.
I was not trying to "conveniently sidestep". To me, that reads like a more emotional wording of the first question you asked, which is if LLM's are expected to improve. To that question I answered yes.
> The assumption that generative AI will improve is as valid as the assumption that it will plateau.
It is certainly not as valid to say that generative AI will plateau. This is comparable to saying that the odds of winning any bet are 50/50, because you either win or lose. Probabilities are a thing. And the probability that the trend will plateau is lower than not.
> It is quite possible that what we are seeing is "as good as it gets", and some major breakthrough, that may or may not happen on our lifetime, is needed.
It's also possible that dolphins are sentient aliens sent here to watch over us.
People invested in something believe that throwing money at it will make it better? Color me shocked.
"eat meat, says the butcher"
The rest of your answer amount to optimistic assumptions that yes, AI future is rosy, based on nothing but a desire that it will, because of course it will.
> This is ego speaking.
That's been my experience of LLM-generated code that people have submitted to open source projects I work on. It's all been crap. Some of it didn't even compile. Some of it changed comments that were previously correct to say something untrue. I've yet to see a single PR that implemented something useful.
I think the problem has become people with no bar for quality submitting PRs that ruin a repo, and then getting mad when it's not merged because ChatGPT reviewed the patch already.
Are you sure it was people? Maybe the AI learned how to make PRs, or is learning how to do so by using your project as a test bed.
Why do we expect that LLMs are going to buck this trend? It's not for accuracy--the previous attempts, when demonstrating their proof-of-concepts, actually reliably worked, whereas with "modern LLMs", virtually every demonstration manages to include "well, okay, the output has a bug here."
LLM's as a product feel practically similar, because _even if_ they could write code that worked in large enough quantities to constitute any decently complex application, the person telling them what problem to solve has to understand the problem space since the LLM's can't reason.
Given that neither of those things are true, it's not much different from visual programming tools, practically speaking.
As soon as you have something less common, it will give you widely incorrect garbage that does not make any sense. Even worse, it APPEARS correct, but it won’t work or will do something else completely.
And I use 4o and o1 every day. Mostly for boilerplate and boring stuff.
I have colleagues that submit ChatGPT generated code and I’ll immediately recognize it because it is just very bad. The colleague would have tested it, so the code does work, but it is always bad, weird or otherwise unusual. Functional, but not nice.
ChatGPT can give very complicated solutions to things that can be solved with a one-liner.
Sure. But the expectation is quantitative improvement - qualitative improvement has not happened, and is unlikely to happen without major research breakthroughs.
LLMs are useful. They still need a lot of supervision & hand holding, and they'll continue to for a long while
And no, it's not "ego speaking". It's long experience. There is fundamentally no reason to believe LLMs will take a leap to "works reliably in subtle circumstances, and will elicit requirements as necessary". (Sure, if you think SWE work is typing keys and make some code, any code, appear, then LLM are a threat)
It would need to become something literally extraterrestrial that has not evolved in the 3.7b+ years prior.
I wouldn't say it's impossible, but if evolution ever got there that creature would be so far removed from a bird that I don't think we'd recognize it :p
That, or they "keep improving every month" til they become evolved enough to build rockets, at which point the entire point of them being birds becomes moot.
Bitcoin was released in what year? I still cannot use it for payments.
No-code solutions exist since when? And still programmers work...
I dont think all hyped techs are fads. For instance: we use SaaS now instead of installing software locally. This transition took the world by storm.
But those tech that needs lots of ads, and lots of zealots, and make incredible promises: they usually are fads.
If anything, I feel like this argument works against you. If people are willing to replace locally installed software with shitty web "apps" that barely compare, why do you think they won't be willing to replace good programmers with LLMs doing a bad job simply because it's trendy?
I love local apps but it's undeniable that developers having to split their attention between platforms lowered quality by a lot. You're probably remembering the exemplars, not the bulk which half-worked and looked bad too
Sure, I can ask for it to write (wrong) boilerplate but it is hardly where work ends. It is up to me to spend the time doing careful due diligence at each and every step. I could ask for it to patch each mistake but, again, it relies on a trained, skillful, many times formally educated domain expert on the other end puppeteering the generative copywriter.
For the many cases where computer programming is similar to writing boilerplate, it could indeed be quite useful but I find the long tail of domain expertises will always be outside the reach of data-driven statistical learners.
https://en.wikipedia.org/wiki/Npm_left-pad_incident
https://old.reddit.com/r/web_design/comments/35prfv/designer...
Much like asking an LLM to solve a problem for you.
You aren't going around applying novel techniques of numerical linear algebra.
Btw. I come from a math background and later went into programming.
Yes. The current technology is at a dead end. The costs for training and for scaling the network are not sustainable. This has been obvious since 2022 and is related to the way in which OpenAI created their product. There is no path described for moving from the current dead end technology to anything that could remotely be described as "AGI."
> This is ego speaking.
This is ignorance manifest.
LLM WILL change the job market dynamics in the coming years. Engineers have been vastly overpaid over the last 10 years. There is no reason to not see a reversal to the mean here. Getting a 500k offer from a FAANG because you studied leetcode for a couple weeks is not going to fly anymore.
What company ever feels they have “enough” engineers? There’s always a massive backlog of work to do. So unless you are just maintaining some frozen legacy system, why would you cut your team size down rather than doubling or tripling your output for the same cost? Especially considering all your competitors have a similar productivity boost available. If your reaction is to cut the team rather than make your product better at a much faster rate (or build more products), you will likely be outcompeted by others willing to make that investment.
In addition to this, most companies aren't willing to give away all off their proprietary IP and knowledge through 3rd party servers.
It will be awhile before engineering jobs are at risk.
Most of the noise I hear about LLMs is about expectations that things will improve. It's always like that: people extrapolate and get money from VCs for that.
The thing is: nobody can know. So when someone extrapolates and says it's likely to happen as they predict, they shouldn't be surprised to get answers that say "I don't have the same beliefs".
It absolutely isn't. I have yet to find an area where LLM-generated code solves the kinds of problems I work on more reliably, effectively, of efficiently than I do.
I'm also not interested in spending my mental energy on code reviews for an uncomphrehending token-prediction golem, let alone finding or fixing bugs in the code it blindly generates. That's a waste of my time and a special kind of personal hell.
I'm a cursor user e.g. and Tab completion is by far the most powerful auto complete I've ever used.
There's scenarios where you can do some major refactors by simply asking (extract this table to its own component while using best react practices to avoid double renders) and it does so istantly. Refactor the tests for it? Again. Semi instant. Meanwhile monocole wielding "senior" is proudly copy pasting, creating files and fixing indentation as I move to the next task.
I don't expect LLMs to do the hard work, but to speed me up.
And people ignoring LLMs are simply slower and less productive.
It's a speed multiplier right now, not a substitute.
If you complain that you don't like the code, you're not understanding the tools nor you can use them, end of story.
I suspect you haven't seen code review at a 500+ seat company.
I'm in my 40s with a pretty interesting background and I feel like maybe I'll make it to retirement. There are still mainframe programmers after all. Maintaining legacy stuff will still have a place.
But I think we'll be the last generation where programmers will be this prevalent and the job market will severely contract. No/low code solutions backed by llms are going to eat away at a lot of what we do, and for traditional programming the tooling we use is going to improve rapidly and greatly reduce the number of developers needed.
Very much so. These things are moving so quickly and agentic systems are already writing complete codebases. Give it a few years. No matter how 1337 you think you are, they are very likely to surpass you in 5-10 years.
Examples?
Anecdotally, I can't recall ever seeing someone on HackerNews accuse LLM's of thinking. This site is probably one of the most educated corners of the internet on the topic.
> They shouldn’t be expected to improve in accuracy because of what they are and how they work.
> And there’s nothing in them that will constrain their token prediction in a way that improves accuracy.
These are both incorrect. LLM's are already quite better today than they were in 2022.
There are definitely people here who think LLMs are conscious.
I mean, in a way, yeah.
Last 10 years were basically one hype-cycle after another filled with lofty predictions that never quite panned out. Besides the fact that many of these predictions kind of fell short, there's also the perception that progress on these various things kind of ground to a halt once the interest faded.
3D printers are interesting. Sure, they have gotten incrementally better after the hype cycle died out, but otherwise their place in society hasn't changed, nor will it likely ever change. It has its utility for prototyping and as a fun hobbyist machine for making plastic toys, but otherwise I remember people saying that we'd be able to just 3D print whatever we needed rather than relying on factories.
Same story with VR. We've made a lot of progress since the first Oculus came out, but otherwise their role in society hasn't changed much since then. The latest VR headsets are still as useless and still as bad for gaming. The metaverse will probably never happen.
With AI, I don't want to be overly dismissive, but at the same time there's a growing consensus that pre-training scaling laws are plateauing, and AI "reasoning" approaches always seemed kind of goofy to me. I wouldn't be surprised if generative AI reaches a kind of equilibrium where it incrementally improves but improves in a way where it gets continuously better at being a junior developer but never quite matures beyond that. The world's smartest beginner if you will.
Which is still pretty significant mind you, it's just that I'm not sure how much this significance will be felt. It's not like one's skillset needs to adjust that much in order to use Cursor or Claude, especially as they get better over time. Even if it made developers 50% more productive, I feel like the impact of this will be balanced-out to a degree by declining interest in programming as a career (feel like coding bootcamp hype has been dead for a while now), a lack of enough young people to replace those that are aging out, the fact that a significant number of developers are, frankly, bad at their job and gave up trying to learn new things a long time ago, etc etc.
I think it really only matters in the end if we actually manage to achieve AGI, once that happens though it'll probably be the end of work and the economy as we know it, so who cares?
I think the other thing to keep in mind is that the history of programming is filled with attempts to basically replace programmers. Prior to generative AI, I remember a lot of noise over low-code / no-code tools, but they were just the latest chapter in the evolution of low-code / no-code. Kind of surprised that even now in Anno Domini 2024 one can make a living developing small-business websites due to the limitations of the latest batch of website builders.
But for VR I think we're still closer to the bottom of the curve - Meta and Valve need something to really sell the technology. The gamble for Valve was that it'd be Half Life: Alyx, and for Meta it was portable VR but the former is too techy to set up (and Half Life is already a nerdy IP) while Meta just doesn't have anything that can convince the average person to get a headset (despite me thinking it's a good value just as a Beat Saber machine). But they're getting there - I've convinced a few friends to get a Quest 3S just to practice piano with Virtuoso and I think it's those kinds of apps I hope we see more of that will bring VR out of the slump.
And then LLMs I think their hype cycle is a lot more elevated since even regular people use them extensively now. There will probably be a crash in terms of experimentation with them but I don't see people stopping their usage and I do see them becoming a lot more useful in the long term - how and when is difficult to predict at the top of the hype curve.
And yet nothing has really changed because he’s still using it to print dumb tchotchkes like every other hobbyist 10 years ago.
I can foresee them getting better but never getting good enough to where they actually fundamentally change society or live up to past promises
> This is ego speaking.
Consider this, 100% of AI training data is human-generated content.
Generally speaking, we apply the 90/10 rule to human generated content: 90% of (books, movies, tv shows, software applications, products available on Amazon) is not very good. 10% shines.
In software development, I would say it's more like 99 to 1 after working in the industry professionally for over 25 years.
How do I divorce this from my personal ego? It's easy to apply objective criteria:
- Is the intent of code easy to understand?
- Are the "moving pieces" isolated, such that you can change the implementation of one with minimal risk of altering the others by mistake?
- Is the solution in code a simple one relative to alternatives?
The majority of human produced code does not pass the above sniff test. Most of my job, as a Principal on a platform team, is cleaning up other peoples' messes and training them how to make less of a mess in the future.
If the majority of human generated content fails to follow basic engineering practices that are employed in other engineering disciplines (i.e: it never ceases to amaze me how much of an uphill battle it is just to get some SWEs just to break down their work into small, single responsibility, easily testable and reusable "modules") then we can't logically expect any better from LLMs because this is what they're being trained on.
And we are VERY far off from LLMs that can weigh the merits of different approaches within the context of the overall business requirements and choose which one makes the most sense for the problem at hand, as opposed to just "what's the most common answer to this question?"
LLMs today are a type of magic trick. You give it a whole bunch of 1s and 0s so that you can input some new 1s and 0s and it can use some fancy proability maths to predict "based on the previous 1s and 0s, what are the statistically most likely next 1s and 0s to follow from the input?"
That is useful, and the result can be shockingly impressive depending on what you're trying to do. But the limitations are so limited that the prospect of replacing an entire high-skilled profession with that magic trick is kind of a joke.
A ton of huge business full of Sr Principal Architect SCRUM masters are about to get disrupted by 80 line ChatGPT wrappers hacked together by a few kids in their dorm room.
Software is interesting because if you buy a refrigerator, even an inexpensive one, you have certain expectations as to its basic functions. If the compressor were to cut out periodically in unexpected ways, affecting your food safety, you would return it.
But in software customers seem to be conditioned to just accept bugs and poor performance as a fact of life.
You're correct that customers don't care about "code quality", because they don't understand code or how to evaluate it.
But you're assuming that customers don't care about the quality of the product they are paying for, and you're divorcing that quality from the quality of the code as if the code doesn't represent THE implementation of the final product. The hardware matters too, but to assume that code quality doesn't directly affect product quality is to pretend that food quality is not directly impacted by its ingredients.
I worked in companies with terrible code, that deployed on an over-engineered cloud provider using custom containers hacked with a nail and a screwdriver, but the product was excellent. Had bugs here and there, but worked and delivered what needs to be delivered.
SWEs need to realize that code doesn't really matter. For 70 years we are debating the best architecture patterns and yet the biggest fear of every developer is working on legacy code, as it's an unmaintainable piece of ... written by humans.
What we need, admittedly, is more research and study around this. I know of one study which supports my position, but I'm happy to admit that the data is sparse.
https://arxiv.org/abs/2203.04374
From the abstract:
"By analyzing activity in 30,737 files, we find that low quality code contains 15 times more defects than high quality code."
Most companies have no relation between their code and their products at all - a major food conglomerate will have hundreds or thousands of IT personnel and no direct link between defects in their business process automation code (which is the #1 employment of developers) and the quality of their products.
For companies where the product does have some tech component (e.g. refrigerators mentioned above) again, I'd bet that most of that companies developers don't work on any code that's intended to be in the product, in such a company there simply is far more programming work outside of that product than inside of one. The companies making a software-first product (like startups on hackernews) where a software defect implies a product defect are an exception, not the mainstream.
Having poor quality code makes refactoring for new features harder, it increases the time to ship and means bugs are harder to fix without side effects.
It also means changes have more side effects and are more likely to contain bugs.
For an MVP or a startup just running off seed funding? Go ham with LLMs and get something in front of your customers, but then when more money is available you need to prioritise making that early code better.
I've seen plenty of companies implode because they fired the one guy that knew their shitty codebase.
The day there is no need to debate systems architecture anymore is the heat death of the universe. Maybe before that AGI will be debating it for us, but it will be debated.
Probably doesn't change the conclusion, but this is no longer the case.
I've yet to try Phi-4, but the whole series has synthetic training data; I don't value Phi-3 despite what the benchmarks claim.
The possible outcome space is not binary (at least in the near term), i.e. either AI replace devs, or it doesn't.
What I'm getting at is this: There's a pervasive attitude among some developers (generally older developers, in my experience) that LLM's are effectively useless. If we're being objective, that is quite plainly not true.
These conversations tend to start out with something like: "Well _my_ work in particular is so complex that LLM's couldn't possibly assist."
As the conversation grows, the tone gradually changes to admitting: "Yes there are some portions of a codebase where LLM's can be helpful, but they can't do _everything_ that an experienced dev does."
It should not even be controversial to say, that AI will only improve at this task. That's what technology does, over the long run.
Fundamentally, there's ego involved whenever someone says "LLM's have _never_ produced useable code." That statement, is provably false.