AI assistants: The convenient crutch for new developers or a recipe for disaster
blog.codeanywhere.com
blog.codeanywhere.com
Now you may be saying "what's the big deal with that? That happens all the time all over the place."
The issue is that the biggest thing AI brings is it allows you to generate lots of code, very fast. So we have more and more code, that is less and less understood by any human being.
Eventually no one has any idea what the code is doing in the first place, and what ends up happening is the Cult Mechanicus of Warhammer 40k we become tech priests running around performing arbitrary and inane rituals to appease "the machine spirit" because no one anywhere has any idea how the dang program actually works.
Like we've all heard horror stories of the Oracle DB code base, imagine if that started to become common. I mean people talk about how AI helps them generate code faster, but to be honest I find most issues are the result of too much code, rather than too little.
My point is I grow concerned about the ability of AI to generate code and the effect that is going to have on increasing the amount of code while reducing understanding.
Like color/exposure auto-balancing tools in video editing. They can be really useful and can often produce solid color/exposure results! But most of the time they only get you 80% of the way or sometimes just spit out really ugly results. Ultimately auto-balancing didn't ruin editing - mediocre editors can't get by on it - so hopefully we don't see that here!
Not unlike the code generated by the fresh out of bootcamp, careless rookie, who isn't trying to make things work so-much-as collect a paycheck. But hell, at this point I need a job too, maybe I should swallow my morals and submit AI solutions as well?
It's like the ostensible toaster company that has outsourced design, engineering, manufacturing, marketing, sales, HR, everything I’ll the way to the janitor. The question then is: what does the toaster company even know about making toasters? If you play this game long enough, then you arrive at a point where entire sectors operate this way. Keep going and the entire economy works this way. At some point we have to stop outsourcing, or we'll lose a lot of knowledge when the original creators of that knowledge kick the bucket. At this point we can't rely on that knowledge being encoded in language models.
If anything, a tool that reveals and punishes incompetence is good for everyone. I'm not the world's most enthusiastic capitalist, but in this case, I think we can let the market sort it out.
The rest of us are in the trenches making do despite the unending stream of spaghetti code that cruel gods have chosen to torment us with.
Somebody please make a SaaS out of this idea, with a fancy tailwind sales page and a “generous free tier”. :-P
maybe an elaborate workflow, but each individual step in that workflow isn't insurmountable
if you cannot get the correct answer in one shot, instead a workflow is required, we should just write off this whole thing as useless or setup that workflow? will let the reader decide
Ever worked with offshored programmers (won't name country or companies)? The things I've seen. Getting code via email that doesn't compile. Then seeing they replied-all, cc'd some random employee not on the project and 2-3 managers about "getting the urgent fix". Having a teenager with a 3 months bootcamp, an AI and a few post-it with git commands would actually be an improvement over this.
> The issue is that the biggest thing AI brings is it allows you to generate lots of code, very fast. So we have more and more code, that is less and less understood by any human being.
I know guys whose whole contracting careers is re-writing (completely from scratch) huge codebase that were offshored. 10x reduction in line count is pretty much standard. I guess these AIs means their business model will keep working for the next decade!
For context, Microsoft introduced a "AI revamped Intellisense" in Visual Studio (not Code) a few months ago. It's not Copilot, it just makes suggestions but only for your next line of code. It's Okish most of the suggestions are good, it saves time on repetitive tasks.
Yesterday I was writing some code to disable some customers. I had a list of all customers and a list of customers to disable. Intellisense offered to pass the list of all customers to the method that will disable them. If I wasn't paying attention, next Monday (and the rest of the week really) would have been quite "entertaining".
There is certainly ongoing research on this, expert systems have been an area of research for a while. Here is recent research relevant to this post: Do Users Write More Insecure Code with AI Assistants? (https://arxiv.org/pdf/2211.03622.pdf)
More relevant discussion, from Andrew Ng's The Batch newsletter: Check the Generated Code Generates Overconfident Coders section: (https://www.deeplearning.ai/the-batch/issue-180/)
For junior developers I wouldn't recommend it, because as you say, they don't have the pattern matching to find and fix the hidden errors that copilot generates. Also, it's harder to conceptualise code that you haven't written.
I would compare it to the way that chess players use engines. Grandmasters can have an engine turned on without it being distracting, and they can tell when the engine is suggesting a good move vs a weird move, and explain what the move accomplishes.
Beginners, are generally advised to play with the engine turned off, and to instead analyse their moves afterward, because otherwise they will just play the engine moves and not learn anything.
“Word processors: a convenient crutch for people who can’t type, or a recipe for disaster?”
God these headlines are going to be hilarious in ten years.
Yes, AI code is primitive today. The first implementations sometimes get basic stuff wrong and often get complicated stuff wrong.
But the state of the art is evolving daily. It’s somewhere between ignorance and gatekeeping to act like the issues we see today (which are already much fewer than we say a year ago) are so endemic to the concept of generated code that the whole thing is a terrible idea.
Most code will be written by AI. You can choose to adapt and leverage that, or you can choose to be that guy who insists that your field is the one true place that we should resist automation (for the good of the world, of course, not any self-interest, that’s just a coincidence).
Relying on perfect coding skills is a losing game. Way better to have healthy code review and testing processes that assume your developers are fallible.
We'll see. I think you're pulling an "apples to oranges" here, comparing the AI situation to the advent of automobiles and computers/word processing.
The big difference is the typical application of the technology. By comparison, AI is absolutely RIFE with fraud & lazy get-rich-quick schemes - on an absolutely unprecedented scale & scope. Just like cryptocurrency: art generation/NFTs, SEO arms race, generated blogs, generated articles, etc. If you look at ChatGPT prompt repositories, these type of prompts are virtually always filling the top several pages when sorting by ratings/views.
I'm more than a little concerned with where things are headed as the internet gets more and more crapped up by AI generated hot-air, riddled with factual inaccuracies. Another concern is, as more companies adopt & rely these tools for regular use, they often downsize their workforces in the process and lose valuable subject matter experts (knowledge) along the way.
There's also the fact that this is going to compound in later training models. I'm not sure where exactly the demarcation point will ultimately lie, but I think we're going to end up thinking of the Internet as two main epochs: "pre-AI" and "post-AI".
IMO: All AI-generated or AI-assisted content should be clearly tagged as such.
In a year or two, leveling content as AI assisted would be like labeling something “electricity assisted” today; you’d just label everything (California prop 65 style).
Everything will have some level of AI involved. I mean even today we have spell checkers and grammar checkers and automatic summarizations that tons of people use. The gradient from that to text expansion and whole document creation is going to be blurry and gradual.
With you on pre-AI and post-AI epochs. It will be sort of like the way the internet itself caused pre-security and post-security epochs, and we’re still dealing with that.
But I think it’s great. Yes, we have to grow up. Yes, our tools have to get better. But just like the internet’s benefits far outweigh the security pain, I feel pretty good about AI cost/benefit ratios.
[1]https://archive.nytimes.com/op-talk.blogs.nytimes.com/2014/0...
[2]https://newlearningonline.com/literacies/chapter-1/socrates-...
In fairness, there are also lots of counterexamples where the long-term consequences of a new technology were not adequately foreseen, and ended up posing huge problems in the long run. So I don't think it's unreasonable for people to look skeptically on these sorts of tools.
Or maybe it’s better to say that hand-wringing is not an argument that merits a counter argument.
Scientific progress doesn't happen by referencing the vast stores of data that we haven't collected.
If someone just asserts some random thing, it’s reasonable to ask for evidence. While their lack of evidence does not disprove their point, it’s certainly an argument for being skeptical about their point.
Otherwise I get to assert that every atom is a superintelligence, and your inability to provide contrary evidence is a point in my favor.
Before we knew about germs the "data" suggested that we could bleed out evil tumors to cure illness, or similarly that the sun revolved around the earth. Would you also have labeled such conjecture as alarmist, knee-jerk hand wringing? A lack of data by no means invalidates a hypothesis, on the contrary it encourages further investigation, which leads to scientific progress.
I found it ironic that you referenced Socrates as an example of such alarmist claims, because the way I interpreted what he said is that without truly understanding a thing, and instead relying upon rote recitation, one could only continue to repeat the conclusions that others had painstakingly worked toward, but they would not be able to explain why or how, only that it is.
You need look no further than the embarrassing circus that was "fact checking" during recent political cycles to see that the concerns Socrates held have in fact come to fruition. Without any evidence except for a tweet hordes of people would confidently and voraciously defend a stance they knew nothing about, often times being completely wrong.
SOCRATES: "The same is true of written words. You’d think they were speaking as if they had some understanding, but if you question anything that has been said because you want to learn more, it continues to signify just that very same thing forever. When it has once been written down, every discourse roams about everywhere, reaching indiscriminately those with understanding no less than those who have no business with it, and it doesn’t know to whom it should speak and to whom it should not."
“Does Visual Studio Rot the Mind?”
http://www.charlespetzold.com/etc/DoesVisualStudioRotTheMind...
Some of it must be rational in some way, but I have to believe a lot of it is insecurity. To be fair, if AI could do what I do better than me, I'd be bummed out because I enjoy doing it and also I'd probably be out of a job. On the other hand, if it's physically possible, then it's inevitable.
There are people who once worried about autocomplete ruining programmers. I'll give you that I don't enjoy programming without intellisense, but it's not because I can't.
You AI assistant may not be able to get everything right within its training data and resource allocation, and you might be too inexperienced to know what it did wrong, but languages and toolchains can adapt to better spot/describe/resolve the characteristic errors of using them. It starts an arms race that ends up with better compilers, linters, debuggers, analyzers, "debugging assistants", etc.
That change ends up being a positive one for everybody, but will take long enough that we will absolutely see plenty of new ugly messes in the meantime.
And plenty of more unnecessary complexity in codebases.
I'm not so sure about that. Idiot-proofing (while being impossible -- ingenious idiots will always find a way around it) comes with tradeoffs that can easily reduce software quality.
I've had good results with code generation for minimal proof-of concept things, like 'show me code for [maze construction algorithm]' or 'visualize this platonic solid in mathplotlib, with numeric labels on every vertex' or 'fetch data from https://hacker-news.firebaseio.com/v0/item/34741898.json and build a pandas dataframe from it'.
I've come to prefer it to visiting Stack Overflow when I need to get up to speed on a new thing, so that I don't have to read people's petty ego trips or irrelevant answers. I also find it responsive when I give carefully detailed prompts to develop an algorithm, as it saves me a lot of typing mistakes and syntactical screwups, eg mixing up dimensions in 3d or flipping [x:] and [:x]. Debugging sometimes feels faster than on my own code, because I didn't mentally commit to the error and create a blind spot for myself.
It's much easier to get good results by specifying incremental (across multiple responses) or stepwise instructions in pseudocode, rather than big bang problem statements. Although the latter occasionally kick out magically-working complete programs, they're much more likely to yield catapults: https://www.youtube.com/watch?v=5aCgSwmm5Ho
I'm certainly guilty of this very same mistake, key word being "mistake." I'd hope for a world where the better way would be to read some good docs which gave an accurate mental model of the thing I want to get up to speed with. You know, organized and structured knowledge.
vs a quick answer to a single point without any context.
I like to go back and forth between the docs and hacking together little things that do one job to maintain a feel for what I'm doing (also in many other contexts besides programming).
This is exactly the same format of argument that convinced me as a learner to use notepad++ for years, which I now believe to be a garbage idea. IDEs provide so much important contextual information - a copilot program to show you different techniques would be helpful in a similar vein.
Caveat: chatGPT is a chat program, that it can produce working code in places is a coincidence, it would not be appropriate for this use
Offtopic, but there was a time when that wasn't an entirely specious argument: back when most major IDEs were very heavyweight and bloated, think Eclipse or Netbeans.
They've improved an enormous amount since then, both in their performance and general UX, as well as in the important assist features they provide, to the point that if you're not using those today, you're definitely hampering your productivity.
There's some relevance to the current situation, which is that these AIs are certainly going to improve, probably very rapidly.
> Caveat: chatGPT is a chat program, that it can produce working code in places is a coincidence
Is it? OpenAI has the Codex models - https://platform.openai.com/docs/models/codex :
> The Codex models are descendants of our GPT-3 models that can understand and generate code. Their training data contains both natural language and billions of lines of public code from GitHub.
...but I don't know how/if those might be integrated into ChatGPT.
From the content creator's perspective, AI tools are of immense help. Finally, we as a small team can allow ourselves to find time and create content instead of being stuck in a constant development loop.
Perhaps you need an editing down phase?
def do_thing():
print('doing a thing')
def do_thing_2():
print('doing a thing 2')
def do_thing_3():
print('doing a thing 3')
def do_all_the_things():
do_thing()
do_thing_2()
do_thing_3()
Where it starts to get a little suspect is in some of its inferences where it doesn't have the localized context or a well known api as its corpus. It can spit out disastrously incorrect code on occasion. Still a time saver, but you have to remain alert.
This is a decent way to check machine learning systems. I haven't tried it in a while, but if you take a text in English and run it through about five different languages sequentially on Google Translate, then go back to English, the result can be interesting.
The author's point might apply to people who are learning a computer language for the very first time. You have to learn the low-level intrinsic complexity by hand, it can't just be dropped into your head in a big chunk. To paraphrase Euclid, "There is no royal road to geometry, or to programming."
Where these tools are pretty great is when having to write something in a new language. For example, if you've never used Javascript but are familiar with C, you might ask, "what's the JS equiavalent of a C struct", or if with Python, "how do I constuct the JS struct equivalent, with its fields being the keys from this Python dict" and then "how do I populate the fields correctly using this json object as the source" and so on.
However, this approach could get one into trouble, if you didn't know what object lifetime was, or what scope was all about, or the difference between a shallow copy and a deep copy, why having many references to a single object might result in data corruption, etc, etc.
Also, often the output or recommendation is just wrong, or very poor code style, so it needs further checking with search engines, usually something like "known issues with library x" etc. That's why I'm not really convinced paying for them, especially on a limited budget, is worth it, at least not quite yet.
Incidentally, I really dislike the subscription model all these online services promote, with no option for just purchasing a service for a given block of time and not having to manually halt the payment.
Where "average" means, imperfect, and somewhere shy of domain expert.
Easy example: "Freelancers paid below prevailing market rate: Convenient crutch or recipe for disaster"
AI bots based on LLM are living mirrors, reflecting the status quo of humanity, and the average human is pretty average, and many are below average and a lot of things.
New developers should avoid crutches as much as possible to begin with. Dependency on such things are, themselves, a recipe for disaster (or at least poor programs), AI or not.
I think one area AI assistants may be very useful is to do standard code quality and advanced linter style code review. I cant remember how many times I have seen engineers give 'cosmetic' CR feedback and completely miss big structural issues. If this makes everybody focus on deeper issues then I am all for it.
I work in embedded software and often I see the code quality coming out of SoC vendors is just full of bad practices. If AI can be a good code acceptance tool especially for last minute fixes we should jump on it.
After just a few minutes, the outputs presented by your AI in a highly confident manner will be shown to be … suspect, or completely wrong.
It quickly casts a shadow over everything else it confidently spews forth as “truth”.
"How can we reasonably verify the output and know it is correct? At least when using AI to create code, we can easily test the code to see if it performs according to the desired behavior. However, that is no easy task for most general information. It is unclear at this point how difficult it will be to improve AI in this regard."
from - https://dakara.substack.com/p/ai-and-the-end-to-all-things
Follow a few people who are proven (by their predictive accuracy) to be experts in their fields.
Their comments on the quality of claims in their field of expertise is much more valuable than the reports being made by "experts" presented on corporate media, or by LLMs.
AI is going to bring about the new era of great centralization. One tool to rule them all. Many concerning issues.
The general populace will use the crippled AIs presented by mainstream corporations and governments.
Anyone who cares about their information will use un-crippled AIs operating without guardrails, which are always available to monitor what they read and hear, and comment on it.
Sort of like when my relatives call me to analyze a potential Phishing email -- I don't do anything difficult; view the raw mail; check the DKIM signing and SPF validity, look at the full URLs of originating servers vs. the claimed From: and the document contents' claimed source, verify all transited MTAs, check the contents for malicious looking payloads, etc.
Any half-decently trained LLM can do all of this, instantly and trivially, and lay out the evidence in an understandable fashion for a non-technical user.
Imagine that, but amplified by an LLM trained to recognize and categorize arguments in a variety of technical fields. Not to tell you what to believe -- but to categorize the claims by its heritage vs. the main groupings of theories in that field, and then give a set of links to highly rated "for" and "against" arguments.
I don't need some LLM trained by a partisan, with guard-rails set up by some corporation's legal and HR departments. I don't think most people will stand for that, when we have laptops that can already run reasonably capable LLMs w/ training on focussed datasets!
Mostly due to the fact the most people will opt for features and convenience over privacy and liberty. The people using Brave, Minds etc will likely use non gated AI, but I suspect the most everyone else will not. Hope I'm wrong.
I used to think you had to be able to work in C to be a "real programmer" and have learned that a lot of good applications have been created by programmers not using C and some that have never used C. Before C it was assembly, and machine code before that.
AI is a tool that all programmers need to learn how to use correctly.