ChatGPT Isn't Coming for Your Coding Job
wired.com
wired.com
There are countless examples of people up-skilling using AI. Today, that might threaten the bottom of the market. Soon, it will put everyone’s jobs at risk for disruption.
[0] https://hbr.org/2023/08/ai-wont-replace-humans-but-humans-wi...
1) Humans are terrible predictors of the future.
2) Linear progression isn’t really a thing in multi step, complex technology progressions
The impact of the steam engine was nuclear weapons. Direct line. Predictable?
Spot on analysis.
"AI"-based static analysis tools might be valuable. If I ever see "AI"-based code generation on a project I run before I get stuck cleaning up the mess.
LLMs don't just dash out hundreds of lines of code. They can take a concept and break it down into concrete steps or even a fully planned architecture for you.
That's literally the hard work you describe. AI can do it beautifully, it just needs a human capable of critical thought to steer it (human understanding of the subject at hand is merely a bonus, not a requirement).
The separation will very clearly be “legacy devs” vs “AI-powered devs”.
I’ve spent a tremendous amount of time thinking about how the problems should be solved on my current project — big picture, and small. Once the system design became clear, about 80% remains as “just write the damn code” type work. I outsource about 70% of that to GPT, while keeping my focus on the important parts; the right abstractions, modular and self-contained functions, good naming and function signatures, making sure it all works and makes sense in harmony.
I don’t care (too much) about the inner workings of every function, so long as the abstraction and usage of it is great (and well tested). Refactoring, if ever needed, is trivial then.
My overall work has never been better.
Anybody dismissing AI for any level of programming is doing themselves a huge disservice.
Could elaborate more on your workflow? Do you use Copilot, or just GPT4? Are you copying and pasting large blocks of existing code to hint at how things should fit together, or do you describe how the existing code is structured in English? Do you find yourself decomposing your work differently so as to fit your new AI workflow?
I talk GPT as if it’s a skilled colleague who’s got memory issues. When it forgets, I remind it with stuff like “this is our current code now, remember X, Y, Z”
I’ve become an even bigger fan of small, compostable functions that do one thing very well. GTP excels at that, both writing and testing them.
I don’t involve it much for architecture and high-level design atm (mostly because I got that part solved on my current project). I tend to have a design in mind, give GPT the overview of how it fits together with mock code, and ask it to review with me, propose other paths we could take, etc before proceeding to implement.
One function at the time, with tests. When refactoring happens, it’s usually isolated to a few hundred lines at most. When tests fail, directly or indirectly, I give it the full output and ask it to debug and fix. I find this helps GPT remember the code’s responsibilities when it gets lost/forgets. It also helps me avoid regressions when GPT returns functions that miss use cases we had covered before.
I keep long running chat threads — weeks at times, hundreds or even thousands of messages. The longer we go, the better it tends to perform (web app performance, even on an M2 Studio, does suffer after a while though)
At worst, GPT is a fantastic rubber duck. At best, it’ll help me see superior approaches and solutions I wouldn’t have considered, AND give me perfect code in seconds.
Once tooling gets really good, and AI can understand the whole code base/database/infra… we’ll probably be in real trouble.
You've really changed your workflow to adapt to these new tools. The amount of mental effort seems comparable to learning a new IDE, maybe a bit less.
Instead of the chat app, I wrote a python script that uses the GPT-4 completion API. I can just pop over to the terminal and type 'chat' and it's there. As far as I can tell, it's basically the same as the app.
We are starting to see AI tooling that can fit an entire 100k line code base in its context window. I still see myself having a job five years out. Ten years out, not so much. Luckily I'll be close to retirement.
I was primarily FE for a while, and used to agonize over the smallest details when building eg React components. After a few years, I concluded that the innards don’t really matter most of the time, at least not early on, so I limited my obsessions to the “public interface” (naming, types and prop design) and libraries of well abstracted, composable, low-level building blocks. If and when things needed to be rewritten or optimized, replace the internals.
ChatGPT just gets me there faster, and more often than not, gives me acceptable production-level innards while I get to stay focused on perfecting the exterior and cohesiveness of the overall solution.
I’ve found this approach usually pays dividend when reworking parts of the system, too — GPT picks up on the flow of everything much better when the code is “self documenting”. Same appears true if humans need to get directly involved.
Still blows my mind on a daily that we’re here already. I’m glad I’m not early in my career, I’d be very worried if I had another 40 years to go.
The PR I'm currently working on has about 1K diffs – I've literally touched NONE of that code directly. Freakin' wild!
I don't think there will be many people not using AI, if at all. Just like everyone learned to use Google and Stackoverflow, everyone will learn to prompt a chatbot. And since it's not really rocket science prompting a chatbot, we might see programming becoming a cheap commodity (and after that - pretty much most knowledge work)
Don’t get me wrong — I think this’ll be the end of MANY jobs within almost every field in the next decades, ours included. But I sure as shit can’t beat it, so may as well join it and get something out of it before I become just a commodity.
Yep that's the crux of the issue I agree. It's not just one repo, any well established org has millions of lines of code - can ChatGPT handle that kind of context size without any human directing it ? It's basically having an LLM trained specifically on your company data, and having it retrained constantly since the data keeps changing - what are the costs of that? We'll see.
GPT-4 and Copilot absolutly makes me more productive as a programmer, wiring all the tests, getting maybe 80% of them right and then I fix them up. I am not producing 10,000 lines of crud, but I AM producing 1000 lines of test-covered code, I'm doing the hard bits, and the AIs are doing the menial parts.
I am by far the most productive programmer on the team however you measure it (we have about 60 different metrics available to management) and a large part of this is be having enough energy to always be attacking the hard problems, because I am not wasting mind power and time writing the trivial bits that can be automated.
Does it get it right every time? Nope. Does it make hilariously wrong mistakes? Yep, about once or twice a day! Is it worth it? Absolutly.
Everyone (including management) is aware of all of the tropes and cliches about how metrics cease to be good when they become targets and how metrics can be gamed blah blah blah - we all know that - and have imbibed that, but the rationale is that more (but imperfect) visibility is better than no visibility or solely subjective interpretations.
As long as all metrics are assumed to be wrong until proven correct, with some digging, some thorough analysis and a judicious and abundant use of the benefit of the doubt, I find them really helpful.
"but this time is different" - I still have not seen a single example that shows otherwise.
don't overthink it
This time is different because AI has the potential to have a similar impact on efficiency across all work. In the past, efficiency gains created totally new spaces of economic activity in which the innovation could not further impact. But AI is a ubiquitous force multiplier, there is no productive human activity that AI can't disrupt. There is no analogous new space of economic activity that humanity as a whole can move to in order to stay relevant to the world's economic activity.
Remember the Luddites. Yea, that group considered to be 'anti-technology', except that isn't what happened. At the time they were kicked to the street to starve with nowhere to live.
AI does not have to replace every job to completely and totally disrupt society in ways like leading to world wars, or the rise again of fascist governments. Blind faith that the magic hand of technology always makes things better is just a form of historical ignorance of all the social changes that had to occur with it to ensure that we were not enslaved by those that owned technology.
I'm kinda sick of the boom and bust and grind. Don't really know where to go next, I don't see that many alternative well-paying jobs out there.
If we lose the developer career path, what the hell is left?
I invest and save aggressively. So a bust cycle lasting a few years won’t matter to me.
Additionally, keep life expanses reasonable and not getting sucked into consumerism keeps the cost of living low.
(a man of your talents?)
The awkward truth is that every other job you can take will appear inefficient. After some time you'll crave a computer to come do aspects of that job too... and hey presto, you're back on a client, coding up some automation feature.
Your subtle observations about inventory management, scheduling and mathematical errors in planning, bugged management so much they took you off 'normal work'. Then you're tasked with automating easy coding tasks that are hard because of outdated company structures, with limited oversight and no fellow bro-grammers to get you out of a jam.
If you do manage to 'lose' a programming job, please tell me how. This career is a one-way ticket to infinity, because you know.
Forget well-paying, I haven no idea what I CAN do that's not programming. Sure I can try working as a teacher or as a nurse - but that's all it will be - trying. I might despair, burn out etc. Even people who are on paper better suited to these jobs burn out all the time. I'm 39 so it's not going to be an easy switch. I hope we all have 5-10 more years in the industry, that's enough time to save enough money.
Probably because the designers of SQL got us as far along towards natural language as they could, and LLMs were built for translation.
However, when gpt gets fluent in formal logic, the number of knowledge workers needed to run a business will shrink tenfold at least.
So what's left of the middle class - we all either go into the trades or to government ?
- Working with unfamiliar APIs which don’t have great documentation. It can be quite helpful for asking questions here, essentially taking the role of Google/StackOverflow, as long as the API was created before the knowledge cutoff and I guess as long as there’s enough content out there. It felt to me like it was able to do quite a good job of linking together the (rather terse) official docs with code from GitHub to show real world examples of how to do stuff. This was definitely net useful for me, but it does like to hallucinate APIs that should exist but don’t.
- Working with unfamiliar languages. I used it to help me write Rust recently (“how can I express this code more succinctly?”, “why won’t this compile?”) and it was quite useful, though I feel like you hit the limits of it if you get deeper into e.g. trying to find a workaround for a borrow checker situation and it’ll just make up code that doesn’t work.
Outside of that, I find it can be quite useful as a supplement to Google, and is good for writing things like regexes, but when I’ve tried doing more advanced coding tasks with GPT-4, I felt like I ended up spending more time trying to make the output compile and work (sometimes without success) than if I had just written it myself. It can be good for working out the broad outline of a solution though.
Overall, I find it a useful tool but I am sceptical of people claiming a 5x increase in productivity thanks to GPT.
As we begin to reshape our approach to project construction, the inclusion of AI systems will become a standard component in our calculations. This realization will inevitably lead us to understand that certain tasks no longer require as many developers as they once did. Consequently, there might be a reduction in hiring rates for developers.
Nevertheless, I remain optimistic that while the demand for developers might become more selective, their compensation could potentially increase due to the specialized skills required in an AI-integrated landscape.
But then came high level assembly languages and even higher level application languages. And easier to use input systems. Programmer used to literally be the person either translating some finished math into assembly language or actually encoding code into punch cards or both.
All of that went away with better tools. So when this article claims that this stuff didn't take jobs, they are wrong. It literally did.
The other thing the article gets wrong is that people are concerned that ChatGPT and the current generation of LLMs will literally replace software developers today in equivalent fully general capacity. No one thinks that. No one whatsoever.
What is obvious, however, is that it is possible to use something like the existing ChatGPT API and models to do certain simple specific programming tasks. That means that _some_ things you might have previously hired a programmer for can now be done automatically.
The applications that I have got working are things like creating a chat interface for querying a specific database, where the system generates the SQL and then formats or summarizes the output. Or creating a web page from a template with just a little bit of custom functionality.
So the current version of GPT cannot replace the actual abilities of a programmer, but these AI-powered app or analysis generator are definitely chipping away at the amount of basic but custom tasks that you need to hire a programmer for.
The main concept though which the author seems to not get is that these systems will continue to improve year in and year out. And the pace we are on, even assuming it slows down a bit, means that we can expect the complexity of programming tasks that LLMs can handle to continue to improve.
There is no reason to believe that it won't pretty shortly end up being better than any human. Especially when you can narrow down the task to a specific popular type of application.
As soon as I saw ChatGPT last year, I realized that this was eventually going to impact jobs for software development. To what degree and how soon was not determined, but it is obvious it will have a very large impact at some point.
So within two days I started working on my own code generation startup leveraging GPT. Within 1-3 years I expect the competition in this area will be very intense. Within five years it will be very obviously competing directly with freelance programmers for small contracts. You will literally see articles like "Use this ChatGPT Plugin instead of spending $10000 on a Freelance Programmer" and it will work.
Actually right now I know there is someone who always advertises on HN for building a GPT-powered chatbot for $10000. There are websites that will make the same thing for under $100 per month. At least one that can generate code for custom chatbots. And I am planning to add chatbot capability to my platform which my code generating AI system can easily hook into. So my system will be able to build custom chatbots also within a few months. As a ChatGPT plugin with hosting for $15/month. Directly competing with that guy charging $10000.
Of course ChatGPT isn't going to replace any programmers now. Is anyone that has actually tried it seriously worried about that?
The reason people are worried is because it provides strong evidence that AGI might actually be possible within a few decades. Before that we really had no idea.
But it clearly isn't there yet.
How tediously derisive. Part of the reason ChatGPT is seminal is that it shows that "next word predictor" can display extremely sophisticated capabilities; more than the sum of its parts. It might not end up in AGI but it's clearly a huge step closer - taking us from "will it every happen?" to "it actually might happen!".
GPT-4 clearly has a type of fairly general purpose intelligence. But there is more than one type of cognitive ability, some which LLMs can't have.
The other person seems to consider AGI to mean something like "living digital person" or whatever.
Bringing up things like sentience and consciousness muddies the waters because nobody knows those even are. HN is full of confident and unfounded assertions that ANNs fundamentally can't be conscious. Or even that they can't reason!
But also even in your statement, the word "level" is problematic. Because there is more than one dimension to the cognition. See Yann LeCun's recent criticisms, many of which are correct. Yet it's clear that LLMs have their own type of often useful and fairly general reasoning.
I know what consciousness, desire and will are because I experience, and act, on them every day. People pointing at things outside themselves to prove intelligence have a much higher bar to clear.
The very idea of intelligence as a discrete quantity is foolhardy and doesn't match what we know to be true in the animal kingdom. Intelligence is deeply rooted in an organism's sensorium.