This is stated as a very matter-of-fact downside, but this is a pretty crazy portent for the future of dev tools / libraries / frameworks / languages.
Predictions:
- LLMs will further amplify the existing winner-take-all, first-mover nature of dev tools
- LLMs will encourage usage of open-source tools because they will be so much more useful with more/better training data
This will be true for people who rely on LLMs to code, which I strongly suggest is not a great long-term bet for a software engineering career.
I know other methods are more efficient. But I’m here to experience life. I want to do the hard things. I want to be uncertain, confused, to make mistakes, and to learn.
AI is a neat tool in limited scopes. Every time I use it though, I feel like I didn’t get the full experience. I’m acutely aware of all the rabbit holes I missed, or the tiny details I’d notice along the way and helpfully remember 10 years later. For everything it adds, it takes at least as much away from where I’m looking. And I like those parts.
All of these have existed in huge numbers for a decade or close to it. This isn’t interesting. It wouldn’t have taken much to do this without AI if these people found motivation through some other means. Nothing notable is happening yet.
That doesn't always matter though; I saw some guy on reddit selling 1000s of subs of a server uptime page they generated with an LLM (Cursor I think it was). Most do it for the money and that is working IF you have the following somewhere.
That's pretty much the same as what you got from old school stackexchange. The LLMs are trained on poor example data (e.g. from stack exchange) and I presume it is a hard problem to filter for just the "good" training data.
The why behind a design decision is something an LLM can't understand simply by being fed the decisions themselves. Hence, though they often stumble into the right answer, they only ever do so because it "seems right based on context," and so they can easily apply a principle correctly one moment and then misuse it the next.
If all a compiler did was spit out a slurry of buggy assembly that misunderstands its context and has to be carefully scrutinized for errors, I would still be writing assembly the old fashioned way.
There is a lot, of course, one can't take 1:1 into the final product, but it helps to find the right libraries, helps to find patterns, the right parts/functions to use where verification in the applicable documentation or source is a lot simpler than finding it in the docs to begin with.
Using it as a tool, while not a source of truth can be good.
And don't get me started about writing all the boilerplate which sometimes is needed, which is too complex for a simple editor shortcut, but too tedious for me as a human. That I review and fix a lot faster than create by hand.
I am currently in the process of hiring a backend engineer. Anybody who does not use AI to aid development work gets an automatic disqualification. In my experience, a good engineer using AI tools will run circles around a good engineer not using AI tools.
AI does help an engineer who embarks on a new voyage through unfamiliar APIs to guide them with usage patterns, but some people become much more efficient by going through the library docs.
Typing out the code is the smallest part of a "good engineer's" job (and even so, having to adapt most of AI generated code is slower than typing it out yourself once you do understand the APIs).
I do think it might work well for MVP-style quick prototyping, but using this as an applicant qualification criteria seems so weird (even when building an MVP, you want some tension between building it quickly and building it the right way).
AIs generate deeply mediocre code. This is better than anything a person who can't code on their own would produce, but an experienced developer will have to spend all their time babysitting the AI to get it to behave properly.
Would you rather write code yourself, or ask a first-year student to write it for you while you watch over their shoulder and tell them to go back and try again every time you notice a mistake? Which of these do you think is faster and better in the long run for the quality of your codebase?
Even supposing that is true, what if you have a choice between a bad engineer who uses AI and a good engineer that won't?
LLMs are often wildly good at being universal translators. So if they pick up general patterns and concepts in popular frameworks, and enough syntax of more niche frameworks, IME, they do a pretty great job of generating good niche framework code.
Similar to how they can talk like a pirate about things things pirates never said in their training data.
If you mean plausible looking code, yes - totally!
If you mean actually usable code: nope. Its always riddled with imaginary library calls that don't exist.
Recently I used Cursor with an in-house developed language and it was able to get almost perfect completions upon seeing ~100 lines of context.
Not all LLMs are equally good at this: Gemini seems to be particularly resistant to adapting to a new syntax
Future frameworks will be designed for AI and enablement. There will be a reversal in convention-over-configuration. Explicit referencing and configuration allow models to make fewer assumptions with less training.
All current models are trained on good and bad examples of existing frameworks. This is why asking an LLM to “code like John Carmack” produces better code.. Future frameworks can quickly build out example documentation and provide it within the framework for AI tools to reference directly.
What I mean is: if you name your modules consistently, say Operation::Object::Verb or Action::ObjectVerb or ObjectManager.doSomething it's really easy for the LLM to guess the next one, just as it is easy for a human.
Add a new file actions/users/update.rb and start typing "Act" and it may guess "class Actions::Users::Update, and start to fill in the code based on nearby modules, switch to the corresponding unit test and it'll fill it in too.
Source: we have our own in-house conventions and it seems copilot gets them right most of the time, ymmv.
Or, taken another way-- developers not using LLMs will be more likely to be first-mover winners.
i'm doing some game development in godot as a hobby, and the current llm's are really bad at it - very often i get code suggestions that use ancient versions of gdscript or the engine. I'd love to have a big enough context window and the tooling needed to go like "look at these godot docs for the current version: (insert link)" and then ask my questions, i think it would fix 99% of these issues. same with other less well-known tools and languages.
- developers will be incapable of writing or debugging code for development stacks or project types without LLMs trained on lots of matching examples