How does AI impact my job as a programmer?
chelseatroy.com
chelseatroy.com
Any programmer worth their salt could have said, and did say, that these tools are incomplete and insufficient, but unfortunately the ones who have the decision making power over such programmers’ “job security” are unable to see how their even knowing what a transistor is will assuage the shareholder’s insatiable need for “growth at all costs!”.
Developers job security should be unchanged but this snake oil is almost tailor made to strip the tech ignorant of all rational faculties.
If and when they do start to ask you back: demand ownership of your work, and triple the salary.
"Our relative lack of skill at investigation becomes clear when we look at the accuracy rate of StackOverflow answers. For the amount of sass you see on that platform, you’d expect the programmers to at least be right. Except they aren’t. We have whole jokes about this too. Again, this is what was used to train the LLMs. Models trained on human data can’t outperform the base error rate in that data."
I think this is an important metric to consider. These LLMs are being trained on data sets with bad results and bad code with no real way to tell the difference. Only someone experienced in this field can read a SO article and appreciate the quality of the responses. An LLM is just summarizing bad information.
In other words, there is no real *intelligence* involved at all.
*Intelligence* has yet to be accurately defined --- and there is no reason to believe it is a statistical function.
In fact, at time of writing, "natural language understanding" is still on the Wikipedia page for AI-complete.
> Intelligence has yet to be accurately defined --- and there is no reason to believe it is a statistical function
Other than the work of Rev. Bayes, Prof. Schrödinger. Plus, now I think about it, saying "randomness can't lead to intelligence" is the talking point for all the people who insist Darwin is wrong and there has to be an intelligent designer.
Evolution is a process where small changes over a long time create new, more complex organisms. LLMs are not that. LLMs are not evolving themselves, giving themselves new abilities. Just adding more variables to a long list of variables is not the same thing.
Given enough time, a monkey randomly typing on a keyboard could hypothetically produce Shakespeare.
But talking points aside, this does not make the monkey more "intelligent" nor does it mean that "randomness" is a practical methodology for achieving specific results on a human timescale.
Particularly when the desired result is as complex and poorly understood as "intelligence".
> But talking points aside, this does not make the monkey more "intelligent" nor does it mean that "randomness" is a practical methodology for achieving specific results on a human timescale.
Not by itself. Evolution is more than just random, it's a specific filter on top of randomness, just like all the other things. And computers roll the dice very quickly compared to reproduction, which is why neither you nor any other human can win a chess game against the best computer any more — it did it on a human timescale.
You cannot make this claim without evidence that the LLM primarily used poor quality SO articles. You don’t include an assessment of all of the other sources. Other sources could include properly functioning open source code, for example.
A couple dozen bad SO articles can easily poison the results of thousands of examples of good OSS code. Code rarely has prose associated with it. SO articles have prose, so these articles will be disproportionately considered as the LLM is self-organizing.
So, it's not necessary that the LLM be trained primarily on bad SO articles for it to have a disproportionate impact on the results it generates from prose prompts.
Do you have any sources related to this disproportionate impact?
I also recommend Mitchell 2023 (doi:10.1073/pnas.2215907120) and Niven 2019 (arXiv:1907.07355) as good starting points.
These don't directly address your question, but within the context of these papers, it's possible to see how the weighting of prose (which is a relation to prompts) and code can become skewed easily.
That doesn’t seem inherently/unavoidably true. Surely a human could read a bunch of SO answers, try them, and thereby outperform the base error rate of the answers.
I don’t see it as impossible for an AI model to exist that does the same.
LLMs are a step on the path toward better human/computer interaction, but they are just a step. Like most new things in the AI field, they have been over-hyped and have moved the market in a giant game of tech FOMO.
For me, the most distressing part is how easily tech people drink the FOMO and then go to work promoting it. This happens over and over again.
The important step is creating an AI model with enough cognition to recognize it needs to do the same before spouting nonsense.
Like literally all powerful tools in the past, ai abuse will come from various directions for various unexpected reasons.
No shit. I lost it when Google AI started recommending people to refill their blinker fluid, which is a meme that's sarcastically been parroted all over the internet for decades but Google's AI took it as serious gospel in their training dataset.
The fact of the matter is these are just LLMs, not AI, they're only as good as their dataset. And since the entire internet is their dataset, along with all the human trolling, sarcasm and shitposting, and without any logic or intuition to distinguish what is sarcasm and what is not, their answers are gonna be questionably acurate a lot of the times when it comes to generic info that every average joe can have an opinion on.
There's no actual intelligence there to call it an AI, it's just a summary of the entire internet, which is definitely useful for some task, but not at replacing human tasks.
I saw some people on HN defended Google AI's shitty answers as being very useful for humor and not for real answers, but then that would be the world's most expensive joke machine for a company who's main business is search and thrived as the go-to for web answers. You don't need to delegate 3% of the world's energy, or whatever it is, towards generating only wrong answers that sound funny.
Somehow key part of IO is missing in this description. For your system to work you have to fit into API/constrains of the system within which yours runs.
> Are large language models gonna cause programmers to lose their jobs? Not anymore than StackOverflow did, in my view. However, it’s going to change them…somewhat.
Disagree.
That will holf true till someone invents 8 concepts using which one can create a whole app just from it's conceptual description. That person would not loose their job, those 9 devs who would have to implement that app otherwise without this appeoach - will.
I forcesee future where 1 team with the right tools replaces today's 10 people. So some will loose their jobs, some will secure them more.
> But like an IDE, or a framework, or a test harness, utility here requires skill on the part of the operator—and not just ChatGPT jockeying skill: programming skill. Existing subject matter expertise.
Agree. You can't estimate quality of the response of the LLM if you don't speak the language they reply at least (or understand nets of concepts hiding behind the words).
---
Also I did not like this passage. The book to my experience is quite useful and is not worthy mentioning in such context.
> First of all, that culture has its infiltrators. We got self-styled prophets writing whole books about their personal philosophies and slapping general-purpose-sounding names on them like “Clean Code.” It reminds me of the dudes who ran around the Middle East 2,000 years ago claiming they personally could introduce you to G-d.
Looks like it's personal attitude towards the author or their work.
---
Other than that it's good solid writing. I enjoyed it.
I see the biggest impact in more traditional startups which involve a lot of manual labour. For us, LLMs are letting us scale much faster than we could before.
For the dev teams I manage, LLMs provide a lot of shortcuts whereas before we spent extensive time hand tuning parsers and simple automations.
Needs also change. Someone somewhere is probably still hiring x86 assembly language programmers for embedded work but they're probably not the highest-paid jobs and you'd likely better be especially expert. And some skills and knowledge just don't matter much any more--at least if they haven't been transferred to newer domains where they can still be applied.Probably, no one want to hire the world's leading consultant on some specific obsolete proprietary architecture.
I use it to help me write code in languages never used before, to manage infrastructure, to summarize manuals, to explain me stuff, and what not. The savings of time and the productivity boost are huge and converts into real income and better understanding of various concepts.
Are the LLM perfect - of course no. Human programmers aren't perfect too, yet they hire them and pay them huge salaries. AI will change a lot of the business and writing essays and making fun on it online won't stop it at all.
However, I think we'll be fine on this front. Smart and able people will find something to do and the really scary consequences of automation-caused job loss are only seen in regions that are disproportionately affected, such as a production plant closing that employs half of an entire town. Programmers are quite wealthy by comparison, which gives them time and opportunities to adapt, and also usually live in big cities, which offer enough different jobs.
What's really scaring me is the recent papers showing AI has theory-of-mind and persuasion capabilities on par with or even surpassing humans. One would think that's the one thing we're actually evolutionarily adapted for...
I guess there was no reason to put in this caveat. Economic developments have never been perfectly fair and neither will this one, which is obvious enough to not need mentioning. The wealth generated by such automation would allow for good social care systems and I hope we get some in any case. That's assuming we don't wipe ourselves out in the process.
Top-tier comedy. When I was in university, not too long ago, all the CS professors were huge fanboys of uncle bob. Lot's of lectures started with a reference to his books, making it seem like a mandatory part of the curriculum.