AI’s coding evolution hinges on collaboration and trust
spectrum.ieee.org
spectrum.ieee.org
I think this is going to make the difference between junior and senior engineers even more drastic than it is today. It's really hard to know what/how to even describe real problems to these tools, and the people who invest the most in their tooling now, are going to be most successful. It's hard for someone who hasn't designed a large codebase already to do this in an ai native way where they don't have the experience of abstracting at the right level and things like that.
Today's equivalent, I've often found some of the best engineers I know have insane setups with nvim or emacs. They invest in their tool chain, and are now bringing AI into.
The hard part is the engineering. Understanding and breaking down the problem, and then actually solving it. If all we gain out of these tools is that we don’t have to write code by hand anymore they are moderately useful but they won’t really be a step change in software development speed.
Maybe if we get super AGI one day. Even then I suspect that from a thermodynamics perspective that might not be cost effective as you often need localized on site intelligence.
It's an interesting question but I bet humans combined with AI tooling will remain cost competitive for a long time barring leaps in say quantum compute. After all organic brains operate at the atomic level already and were honed in an extremely competitive environment for billions of years. The calories and resources required to create highly efficient massively powerful neural compute had incredibly thin resource "margins" with huge advantages for species to utilize.
What’s hard is coming up with the algorithm/system design, making the right choices that will scale and won’t become a maintenance nightmare, etc. And yeah, after almost a decade, I have picked up enough I can at least write an outline of a solution that will work alright. But there are still so many tricky edge cases and scaling problems that make it hard to turn “alright” into “really good!”
Sure, AI can help… but it mostly helps with greenfield projects. It doesn’t know about the conversations on slack & jira from a year ago. It doesn’t know about the dozens of other systems and ways the project interacts with other parts of the business. It doesn’t know why whatever regurgitated approach won’t be a good fit for our specific use case. And elaborating all of that detail is not easy! Part of what makes you a good employee is the shit you picked up over the past several months & years that is joe ingrained in your mind when you start working on new projects.
I would argue that if you can't describe the problem in plain language then you don't have a very good chance of solving it with code or otherwise.
Personally I find that the act of describing the problem will often reveal a good solution...then it's just a matter of seeing if the LLM agrees with me or if it has a difference idea (for better or worse).
Oh, I get it. They're saying you should be able to write it in C.
[Jokes aside, I would be interested in hearing from bridge builders, aerospace engineers, or nuclear scientists. Pretty sure they're using math and not 'plain language'.]
But the sneakier part of the problem is that as business rules get more complex it's usually harder to completely describe the problem in plain language than in a more formally specified language. For instance, plain-language "apply the user's promo code" doesn't capture the nasty if/else tree you might hit when you're deep in the codebase and see that there's already a bunch of restrictions on other types of promo codes and the product manager didn't think about which of those restrictions should apply to this new promo code. And at this point you're gonna need to use plain language to describe and refine the problem with the product manager - but if you instead were relying on an LLM to turn your short sentence into the right code, it might pick the wrong thing when it comes to modifying that existing code.
Sure, but you've already lost at this point. Your code is so jacked up that you can't even describe what it does in words.
Now imagine what happens when a customer calls customer service and asks why their promo code isn't working.
> And at this point you're gonna need to use plain language to describe and refine the problem with the product manager - but if you instead were relying on an LLM to turn your short sentence into the right code, it might pick the wrong thing when it comes to modifying that existing code.
I think you're agreeing with me here...
But TBH this all feels like the establishment trying to protect it's turf, which is understandable. But using the argument of "my code is so overcomplicated and confusing that nobody but me can understand it but me" probably isn't the best position to take.
I find this likely, but totally irrelevant to their success now and in the future. AI tools are as much of a trivial choice as any other text editor features. The resulting time spent and code quality are the same regardless of personal preferences.
One weakness of coding agents is that sometimes all it sees are the codes, and not the outputs. That's why I've been working on agent instructions/tools/MCP servers that empower it with all the same access that I have. For example, this is a custom chat mode for GitHub Copilot in VS Code: https://raw.githubusercontent.com/Azure-Samples/azure-search...
I give it access to run code, run tests and see the output, run the local server and see the output, and use the Playwright MCP tools on that local server. That gives the agent almost every ability that I have - the only tool that it lacks is the breakpoint debugger, as that is not yet exposed to Copilot. I'm hoping it will be in the future, as it would be very interesting to see how an agent would step through and inspect variables.
I've had a lot more success when I actively customize the agent's environment, and then I can collaborate more easily with it.
gpt-5 was created to be able to service 200m daily active users.
Because they already did try making a much larger, more expensive model, it was called GPT-4.5. It failed, it wasn't actually that much smarter despite being insanely expensive, and they retired it after a few months.
It's horribly outdated way of thinking that an singular AI entity would be able to handle all stacks all problems directed at it because no developer is using it that way.
AI is a great tool for both coders and artists and these outlandish titles that grab attention really seem to be echo chambers aimed at people who are convinced that AI isn't going to replace them which is true but the opposite is also true.
I imagine there's a big difference in using AI for building, say, an online forum vs. building a flight control system, both in terms of what the AI theoretically can do, and in terms of what we maybe should or should not be letting the AI do.
With the recent models its now encroaching similarly on all fronts, I think the next few iterations we'll see LLM solidify itself as a meta-compiler that will be deployed locally for more FCS type systems.
At the end of the day the hazards are still same with or without AI, you need checks and bounds, you need proper vetting of code and quality but overall it probably doesn't make sense to charge an hourly rate because an AI would drastically cut down such billing schemes.
For me "replacement" is largely a 70~80% reduction in either hourly wages, job positions or both and from the job market data I see it can get there.
I also write high performance Go server code, where it works a lot less well. It doesn't follow my rules for ptr APIs or using sync mutexes or atomic operations across a code base. It (probably slightly older version than SOTA) didn't read deep call chains accurately for refactoring. It's still worth trying but if that was my sole work it would probably not be worth it.
On the other hand for personal productivity, emacs functions and config, getting a good .mcp.json, it is also very good and generates code that partakes in the exponential growth of good code. (Unlike data viz where there is a tendency to build something and then the utility declines over time).
And until the day that humans are no longer driving the bus that will remain the case.
Is it? I can, in most traditional programming languages commonly used to today, using decades old compiler technology, say something like "x = [1,2,3]" and it will, for example, systematically generate all the code necessary to allocate memory without any need for me to be any more explicit about it. It would be fair to say AI offers an even higher level abstraction, like how most programming languages used today are a higher level abstraction over assembly, but fundamentally different it is not.
"generate a c program that uses gcc 128 bit floats and systematically generates all quadratic roots in order of the max size of their minimal polynomial coefficients, and then sort them and calculate the distribution of the intervals between adjacent numbers" is just code. You still have to write the code get AI to translate it into a lower-level abstraction. It doesn't magically go off and do its own autonomous thing.
Imagine driving your car, you turn right, but today turning right slams on the brakes, and 10 people rear end you! That's current AI.
Amazing how someone writing for an IEEE website can't keep their eyes on the fundamentals.
We'll reap the productivity benefits from this new tool, create more work for ourselves, output will stabilize at a new level and salaries will stagnate again, as it always happens.
Everyday, I see ads on YouTube with smooth-talking, real-looking AI-generated actors. Each one represents one less person that would have been paid.
There is no exact measure of correctness in design; one bad bit does not stop the show. The clients don't even want real art. Artists sometimes refer to commercial work as "selling out", referring to hanging their artistic integrity on the hook to make a living. Now "selling out" competes with AI which has no artistic integrity to hang on the hook, works 24 hours a day for peanuts and is astonishingly prolific.
You can argue that is a bad thing (local designers/content producers/actors/etc lost revenue, while the money was sent to $BigTech) or that this was a good thing (lower cost to make ad means taxpayer money saved, paying $BigTech has lower chance of corruption vs hiring local marketing firm - which is very common here).
[1]https://www.cnnbrasil.com.br/tecnologia/video-feito-com-inte...
Of course that line of reasoning reduces similar to other automation / minimum wage / etc discussions
The thing is that they would not have paid for the actor anyway. It’s that having an “actor” and special effects for your ads cost nothing, so why not?
The quality of their ads went up, the money changing hands did not change.
Were AI-generated actors chosen over real actors, or was the alternative using some other low-cost method for an advertisement like just colorful words moving around on a screen? Or the ad not being made at all?
The existence of ads using generative AI "actors" doesn't prove that an actor wasn't paid. This is the same logical fallacy as claiming that one pirated copy of software represents a lost sale.
B) Even in cases where AI actors are used where there wouldn’t have been actors before, the skillset is still devalued, and even that modest insulation for higher-end work is almost certainly temporary. Someone doing a worse version of what you do for 1% of your pay affects the market, and saving 99% is great incentive for companies to change their strategy until the worse version is good enough.
Probably took me the same amount of time to generate a pleasing video as I would have spent browsing Shutterstock. Only difference is my money goes to one corporation instead of the other.
As far as the video is concerned, it adds a bit of a wow factor to get people interested, but ultimately it's the same old graphs and bullet points with words and numbers that matter. And those could just as well have been done on an overhead transparency in 1987.
Among other things, this will remove most entry-level jobs, making senior-level actors more rare and expensive.
You can still find a tailor who will hand make you a bespoke clothing or sew your own clothes yourself (as even the boomer generation often did growing up), but tailored clothing is a tiny fraction of the amount of clothing in circulation. Do tailors and artisanal cheese makers still exist? Yep, they are not extinct. But they are hugely marginalized compared to machine-made alternatives.
In the time before automated T-shirt production, almost nobody bought clothes. They were just far too expensive. There were of course people that did. And those paid extremely well. But those kinds of tailors still exist!
At the same time, I do think that the comparison is less than apt. And a better one would be comparing it to the fate of lectors and copywriters. A significant amount of those have been superseded by spellchecking tools or will be by AI “reformulations”.
Yet even here I’m not sure if those jobs have seen a significant decline in absolute numbers. Even while their relative frequency kind if obviously tends to 0
Use of AI is exerting a downward pressure on artists and designers to get paid.
It's not true that AI is only servicing the pent-up demand for that kind of work from clients who would never have paid people to do it.
When GPT3.5 first landed a lifelong writer/editor saw a steep decrease in jobs. A year later the jobs changed to "can you edit this AI generated text to sound human", and now they continue to work doing normal editing for human or human-ish writing while declining the slop-correction deluge because it is terrible work.
I can't help but see the software analogy for this.
How long do you suppose it will be before we can't tell the difference between it and reality anymore? A few years at the most. Then what?
Just wait; the stuff is coming. Ultra-realistic full-length feature films with compelling AI characters that are not consistent from beginning to end, but appear in multiple features.
The public will swallow it up.
or equally, one more advert which (let's say rightly) wouldn't have been made.
seriously though, automation allows us to do things that would not have been possible or affordable before. some of these are good things.
For cheap stuff it’s true. However, nobody wants to watch or listen generated content and this will wear thin aside from the niche it’ll take hold of and permanently replace humans.
It took under a decade to get AI to this stage - where it can build small scripts and tiny services entirely on its own. I see no fundamental limitations that would prevent further improvements. I see no reason why it would stop at human level of performance either.
haha, well said, I've got to remember that one. HN is a smelly place when it comes to AI coping.
Now, there’s a little room between the two—maybe the site is full of coders on a cope train, hoping that we’ll be empowered by nice little tools rather than totally replaced. And, ya know, multiple posters with multiple opinions, some contradictions are expected.
But I do find it pretty funny to see the multiple posters here describe the site they are using as suffering from multiple, contradictory, glaringly obvious blindspots.
the article's subtitle is currently false, people collaborate more with the works of others through these systems and it would be extremely difficult to incentivize any equally signifciant number of the enterprise software shops, numerics labs, etc to share code: even joint ventures like Accenture do not scrape all their own private repos and report their patterns back to Microsoft every time they re-implement the same .NET systems over and over
It’s not under a decade for ai to get to this stage but multiple decades of work, with algorithms finally able to take advantage of gpu hardware to massively excel.
There’s already feeling that growth has slowed, I’m not seeing the rise in performance at coding tasks that I saw over the past few years. I see no fundamental improvements that would suggest exponential growth or human level of performance.
All you really need is for performance to keep increasing steadily at a good rate.
If the exponential growth tops out, and AI only gains a linear two days per year of "task-completion time horizon" once it does? It'll be able to complete a small scrum sprint autonomously by year 2035. Edging more and more into the "seasoned professional developer" territory with each passing year, little by little.
If we don't see some serious fencing, I will not be surprised by some spectacular AI-caused failures in the next 3 years that wipe out companies.
Business typically follows a risk-based approach to things, and in this case entire industries are yolo'ing.
ok, ok! just like you can find for much less computation power involved using a search engine, forums/websites having if not your question, something similar or a snippet [0] helping you solve your doubt... all of that free of tokens and companies profiting over what the internet have build! even FOSS generative AI can give billions USD to GPU manufacturers
[0] just a silly script that can lead a bunch of logic: https://stackoverflow.com/questions/70058132/how-do-i-make-a...
How about the fact that AI is only trained to complete text and literally has no "mind" within which to conceive or reason about concepts? Fundamentally, it is only trained to sound like a human.
An LLM base model isn't trained for abstract thinking, but it still ends up developing abstract thinking internally - because that's the easiest way for it to mimic the breadth and depth of the training data. All LLMs operate in abstracts, using the same manner of informal reasoning as humans do. Even the mistakes they make are amusingly humanlike.
There's no part of an LLM that's called a "mind", but it has a "forward pass", which is quite similar in function. An LLM reasons in small slices - elevating its input text to a highly abstract representation, and then reducing it back down to a token prediction logit, one token at a time.
This has been demonstrated so many times.
They don’t make mistakes. It doesn’t make any sense to claim they do because their goal is simply to produce a statistically likely output. Whether or not that output is correct outside of their universe is not relevant.
What you’re doing is anthropomorphizing them and then trying to explain your observations in that context. The problem is that doesn’t make any sense.
Those are real examples of the kind of thing that can be found in modern production grade AIs. Not "anthropomorphizing" means not understanding how modern AI operates at all.
You've clearly read a lot of social media content about AI, but have you ever read any philosophy?
Anything that actually works and is in any way useful is removed from philosophy and gets its own field. So philosophy is left as, largely, a collection of curios and failures.
Also, I would advise you to never discuss philosophy with an LLM. It might be a legitimate cognitohazard.
Not to mention the effect of formal logic in computer science
If you don't have anything measurable, you don't have anything at all. And philosophy doesn't deal in measurables.
You're not being serious.
LLM's do not act entirely like a human. If they did, we'd be celebrating AGI!
Don't expect them to show mastery of spatial reasoning or agentic behavior or physical dexterity out of the box.
They still capture enough humanlike behavior to yield the most general AI systems ever built.
How can you say this when progress has so clearly stagnated already? The past year has been nothing but marginal improvements at best, culminating in GPT-5 which can barely be considered an upgrade over 4o in terms of pure intelligence despite the significant connotation attached to the number.
Even o1 was a major, groundbreaking upgrade over 4o. RLVR with CoT reasoning opened up an entire new dimension of performance scaling. And o1 is, in turn, already obsoleted - first by o3, and then by GPT-5.
There is zero reason to think AI is some exception that will continue to exponentially improve without limit. We already seem to be at the point of diminishing returns. Sinking absurd amounts of money and resources to train models that show incremental improvements.
To get this far they have had to spend hundreds of billions and have used up the majority of the data they have access to. We are at the point of trying to train AI on generated data and hoping that it doesn’t just cause the entire thing degrade.
I suspect once you have studied how we actually got to where we are today, you might see why your lack of seeing any limitations may not be the flex you think it is.
Proofreaders still exist, despite spell checker. Art assistants still exist, despite Photoshop. There's always more work to do, you just incorporate the new tools and bump the productivity, until it gets so commoditized it stops being a competitive advantage.
Saying AI "replaces" anyone is just a matter of rhetoric to justify lower salaries, as always.
You cannot use just a spell checker to write a book (no matter how bad) or photoshop (non-AI) plugins to automatically create meaningful artwork, replacing human intervention.
Business people "up the ladder" are already threatening with reducing the workforce and firing people because they can (allegedly) be replaced by AI. No writer was ever threatened by a spellchecker.
Hollywood studio execs are putting pressure on writers, and now they can leverage AI as yet another tool against them.
The thing is trained on heaps and heaps of human output. You better anthropomorphize if you want to stay ahead of the curve.
Heck, people literally used to think eyes are the source of light since everything is dark when you close them.
People are immensely, incredibly, unimaginably stupid. It has taken a lot of miracles put together to get us where we are now…but the fundamentals of what we are haven’t changed.
So what you are saying is that beings without a central nervous system cannot experience "excitement"?
or perhaps the meaning of too many words has changed, and their context. When Hippocrates claimed that the brain was an organ to cool the blood, perhaps he meant that we use our thought to temper our emotions, i.e. what he said agrees with our modern understanding.
However, many people read Hippocrates and laugh at him, because they think he meant the brain was some kind of radiator.
Maybe because we stopped talking about "excitable" people as being "hot-blooded"
The belief that the heart was the seat of thought and emotion was shared by numerous cultures[0], and was based on their naive interpretation of physiology and biology and cannot be dismissed as a modern misinterpretation of a single vague aphorism by a single person due to the preponderance of documentary evidence to the contrary from contemporary sources. Also, you're probably talking about Aristotle, not Hippocrates.
>Maybe because we stopped talking about "excitable" people as being "hot-blooded"
Also people still say "hot blooded" all the time.
Stupid is people in 2025 believing the world is flat and germ theory is a hoax. Ignorance becomes stupidity when our species stands on the shoulders of giants but some people simply refuse to open their eyes.
Of course, all these words have some overlap. My larger point is, people rarely come to rational conclusions organically, and it takes decades to centuries for even the most empirically verifiable idea to permeate, especially in the face of misinformation campaigns or when against “common sense”.
I really like the word "assistant" for what we have today. The AI code assistant tools available today, like Claude Code and GitHub Copilot, can't replace humans in doing software development. Not even close. But they are often useful to human developers, and today, that's the more important measure.
I've been spending time with various AI tools, especially Claude Code and GitHub Copilot. They're amazing one minute, and they make bone-headedly bad recommendations the next. It takes effort to learn how to create good prompts, and if you want the results to be good, you have to review and critique the results. I'm particularly concerned about security. They're definitely happy to write insecure code. If you know what you're doing, prompt them well, and review their results, you can get good results.
I don't know if they'll ever reach "full autonomy". They don't need to get there to be useful.
They do, unless you just want to babysit them all the time
Maybe babysitting a bad machine sounds great to you, it sounds like the torment nexus to me
I've just tried to get ChatGPT to help me generate a plumbing part that I will 3D print. Bit bizarre but the part in question (which is currently cracked) holds the other part in place and that provides the water seal with two O rings.
I got so far with OpenSCAD. I started off with describing a part that Chat came back with as a "cup". Fair enough and so far so magical. Then I asked it to put a helical screw thread on the inside of the part. It suggested a popular library for this (BOSL2) and then got things a bit wrong.
This is how it goes: Chat int al will get you close but as things get more specialized it will start to go off piste and tumble.
I own a couple of slide rules. If I read them incorrectly then I will get a calculation wrong. Chat n that are just very fancy slide rules and I still find them fabulous.
How is this any different than the way I program?
I've contributed genuinely useful features to FLOSS projects "as well as irrelevant suggestions or code fixes with subtle problems", mostly the latter as there was always a few stages of improvement and/or finishing by the core devs of the program I used to haunt. Honestly, I was less than half as useful as the current crop of robots and they still tolerated (in fact, encouraged) my involvement.
Don’t sell yourself short. External contributors are extremely valuable as they are often users and provide real world validation of a need for whatever they are contributing. They also retain any knowledge for any feedback they receive that they can apply to future contributions. And they also become advocates for that software, helping it grow its user base.
LLMs, not so much.
I personally haven't written any significant code by hand since claude code landed. I also have a high tolerance for prompting and re-prompting. Some of my colleagues would get upset if it wasn't mostly one shotting issues and had a really low tolerance for it going off the rails.
Since gpt-5-high came out, I rarely have to re-prompt. Strong CI pipeline and well defined AGENTS.md goes an incredibly long way.
AI code tab-complete is fantastic. It's at least an order of magnitude more powerful than IDE-assisted auto refactors.
AI graphics design tools are probably the single best thing in the field. Editing photos, creating new graphics, making marketing materials, shooting and editing videos is now extremely easy. It's a 100x speed up and a 1000x cost reduction. You still have to re-roll the generations repeatedly, but with a competent editing tool you can speed run any design work. This is one area where non-experts can also use the tools.
These tools seem great because they are less sensitive than humans to the mess and lift us over the tedious work. But at the same time, they're giving us an excuse to not fix what needed to be fixed and, in doing so, they're adding more crap to the heap.
Maybe what we need is forcing the tools to build on a simpler base, so we can keep an understanding of the results.
https://www.comp.nus.edu.sg/cs/people/abhik/
Of course, he also had a startup for an agentic "AI" approach for software.
I bet he hasn't written any real software like many academics. We should not listen to these people who chair conferences and let them dictate the future of real hackers.
AI is definitely not ready for an Engineering role. My recent experience with ChatGPT5(Preview) via Visual Studio Code tells me that it might perform acceptably as a junior programmer. However, because I'm an old retired self taught programmer who only ever managed 1 other programmer, I lack the experience to know what's acceptable as a Junior Programmer at FAANG and elsewhere.
Share what you built and how you prompted and what you are making from it, and how many tokens you paid to rummage through.
I'm trying to build a software stack that can eventually take something like a PyTorch model, and unwind everything, resulting in a directed acyclic graph of individual bit-level operations (OR, AND, XOR). That graph will then be compiled into a bitstream suitable for an FPGA-like substrate that eliminates the memory/compute divide, the BitGrid[1].
FPGA routing is a non-trivial problem, I'm hoping to get it down to seconds. I'm currently trying to build the software stack to make it usable.
The goal is to answer questions about BitGrid:
How efficiently can I pack a program into the hardware?
Is the model I've chosen for a cell optimal?
How many femtojoules per operation would a cell actually take?
This isn't just some CRUD application with a web front end. There are a number of layers of abstraction at play, and the LLMs seem to handle it well if you limit the depth under consideration.
[1] BitGrid eliminates the traditional memory/compute divide that causes most of the energy consumption of CPUs, GPUs, and other accelerators. Even FPGA systems tend to focus on emulation of these models, and routing fabric for minimum latency, instead of maximum performance. Because all the active lines only reach nearest neighbors, power consumption for a given operation can be far lower than the traditional approach.
PS: I pay $10/month for GitHub CoPilot, which apparently now includes ChatGPT5
You're making a distinction that might be interesting in some esoteric sense, but that doesn't exist in the industry. When it comes to software, the architects are also the construction crew. There's nothing to confuse if the terms are effectively synonymous.
Sure it exists. Not everywhere, not for all software, but it does exist.
Just be aware that AI is a tool not a replacement but a human apt at AI as a tool will replace the former.
But there is a "sweet spot" where it's amazing, specifically highly targeted tasks with a specific context. I wanted a simple media converter app that tied into ffmpeg, and I didn't want to install any of the spammy or bloated options I found... so I got Claude to build one. It took about 30 minutes and works great.
I also asked it to update some legacy project, and it fell down a testing a loop where it failed to understand the testing database was missing. Had I not had years of knowledge, I would've looked at the output and suggestions Claude was giving and spent hours on it... but it was a simple command that fixed it. As with all new tech, your milage will vary.
That is, you have some context, ie the prompt and any other text, and the LLM produces a plausible continuation or alteration of that prompt and text.
My intuition leads me to a thought like: To progress, the context must compress into a fractal representation.
I feel very confident that someone smarter and MUCH better paid than me is already working on this.
A big part of my career has been the modification of enterprise software to fit a company's needs. Rarely was any one addition more than a few hundred lines of code. I can see a future where there will be simple options for a non-coder to add to an app.
True, it's not a coder, but that doesn't mean it won't fundamentally change how apps are made and it will reduce the number of master programmers needed. It won't replace all programmers, but it will greatly reduce the number that are needed, which country they work in and the language they use to program apps.
Programming has mainly been a career that requires the individual to understand English. That is changing. I can see a future where code can be created in multiple human languages. Programming was well-paid because relatively few people had the expertise to do it. That won't be the case, and the pay will adjust downward as needed. AI might not be a coder, but it will let many more people become coders. In the future, coding will be in the same pay range as clerical work. Companies will be hiring Programming Clerks rather than Programming Engineers.
Disagree, programming is a career where in order to be good you can
1. Break down a big problem into smaller ones, creating abstractions
2. Implement those abstractions one by one to end up with a full program
3. Refactor those abstractions if requirements change or (better) reimplement an abstraction a different way
You do all of this to make complex software digestible by a human, so that they don't have to have the entire system 'in context'.
This prophesied view of software development will mean you end up with code that's likely only maintainable by the model itself.
I can't imagine the vendor lock in of that.... You have the source, but it is in such a state that no human can maintain it?
It’s much worse than that.
What happens when the erroneous output caused by model blind spots gets fed back into the model?
Those blind spots get reinforced.
Doesn’t matter how small that error rate is (and it’s not small). The errors will compound.
Vendor lock-in won’t matter because it will simply stop working/become totally unrecoverable.
Years ago, I supported a team of finance professionals who were largely quite competent at coding but knew nothing about software engineering. They had thousands of scripts and spreadsheets: they used version control, but kept separate long-lived branches for client-specific variations of different models. There were no tests for anything; half the tools would break when the clocks changed.
They weren't dumb, but their incentives weren't about building anything we might recognize as an engineered application. I suspect something similar will happen turning end users loose with AI.
If it's taking you that long to direct the AI, then either you're throwing too small a problem at it, or too big a problem at it, or you're not directing its attention properly.
In a way, your prompts should feel like writing user documentation:
Refactor all of the decode functions in decoder.rs to return the number of bytes decoded
in addition to the decoded values they already return. While refactoring, follow these principles:
* Use established best practices and choose the most idiomatic approaches.
* Avoid using clone or unwrap.
* Update the unit tests to account for the extra return values, and make sure the tests check the additional return values.
When you're finished, run clippy and fix any issues it finds. Then run rustfmt on the crate.Talking to a chat bot like you'd talk to a human is a quick way to be disappointed. They don't work that way, despite sometimes sounding like they do.
But the end result is the same.
They fix a problem: the mutual incompatibility between software. "Free software" is not the answer, as it's used today: for example, the canonical Free Software OS, Linux, is fragmented into many mutually incompatible ABIs.
We need to tear down barriers between software. Barriers between languages, frameworks and OS ABIs. I don't have a good plan , but think it is the true solution. LLMs paper over these incompatibilities, and in doing so, provide value. Imperfect, but good enough until we destroy the barriers properly.
Example: Why am I using an LLM for my rust code? Because it is taking code that solves my problem, but in Python, C, or papers, and translating in a way compatible with the rust dialect I use. The problem is solved multiple times, but not in a way I can use without a translation layer. LLMs provide that.
That's the big thing that's warmed up to programming languages with good macro systems in the past few years (and I absolutely LOVE Zig's comptime paradigm). At the same time, I've got serious questions about the programmers who dismiss macros as overkill for their needs, but then immediately pull out the LLM.
But I've wondered the same thing. On some level, the coding superpowers allegedly wrought by LLMs seem reminiscent of proficient use of macros. But most programmers don't use them, and most programming languages don't support them.
What feels primitive to me is how we approach programming in industry as a process of trial and error rather than one of rigour.
These are tools that automate copy-pasting from Stack Overflow and GitHub, running tools, and generating a ton of noise to sift through. They hallucinate code, documentation, and various other artifacts that are sometimes useful and are occasionally complete BS.
Some people find that they can make useful tools out of these things. Great.
A real programmer is still a human.
Update: nothing wrong with trial and error as a process. I use it a lot. But there are lots of places where we use this method that seem inappropriate and sometimes even dangerous. Yet it’s the most common tool we have and everything starts to look like a nail.
Context doesn't work the same way as memory + experience in humans. While humans have an impression and a flexible mental model of any single domain, AI needs hard data, which is hard to manage with context and can't really be worked around by fine-tuning in practice, lest you have to retrain the model on each and every code merge.
Also, not every project is highly complex or involves huge codebases and, for those that are, a junior developer might be assigned to a reasonably self contained and small module within the project (i.e. something that a LLM can do better at).
Finally, although this wasn't part of my original argument: before modern LLMs, let's say that a senior developer could produce N units of work per Y units of time. Now, with the help of a LLM, they are able to produce XN units of work per Y unit of time. I don't think it's too controversial to claim that some portion of developers are now more productive than they were before. And if developers in general are more productive, then you can expect some reduction in the number of developers that you need for a given project. So, in an indirect sense, LLMs are already capable of replacing coders.
This pattern I see play out time and time again. Senior builds the integration to various team services, then juniors tweak & mimic those to do more.
> let's say that a senior developer could produce N units of work per Y units of time.
So, it's more the case that senior developers are building full production assembly lines, that can then be replicated to do similar things relatively easily. The cost of the units of work are not at all equal.
How does this apply to AI? Good luck getting AI to do something novel, by definition it can't. OTOH, extending an existing pattern is trivial for a senior developer to do after they have built that pattern; juniors spend a lot of time at it cause they have to learn everything.
So, who is the AI going to replace? If the junior, then you have a senior developer using an AI to do something that is already trivial for them to do. No way AI is replacing the senior.. So who? Who is getting replaced by the AI and who is using it?
uhhh, not sure even the best people or teams are very good at this either. Condemning AI for not being capable of something we're not capable of, ok...
“If it takes longer to explain to the system all the things you want to do and all the details of what you want to do, then all you have is just programming by another name.”
This is called the specification process, which hopefully is already occurring today.
There's so much self-serving bias in articles like this, as well as the comments on HN, Reddit, etc. It's good to critique AI, but that self-serving line is frequently crossed by many people.