Hot takes on Devin, the AI software engineer
tomhipwell.co
tomhipwell.co
Now it's a near fully-functional teammate that needs a bit of supervision -- still no one complained because it needs human team members to instruct it
Next it will be a full-functional teammate that is probably as good as a junior coder -- but no one will complain because senior coders will get paid more and companies will have to hire less
Then, there will be the expert AI coder, but still no one will complain because we will need system architects to design the system that the AI codes.
All along, no one will complain because those with jobs will still have them and those without will be too busy thinking about how to provide for their families. Get another job? Oh yeah, AI took that one too.
All along, we are incrementally improving AI because it is an intellectual amusement -- we simply never take into account the social consequences. This path of destruction is so blindingly obvious, it's incredible that everyone is still working on AI instead of trying to destroy it. But wait, I get it, those who jump on earliest still have a chance at a bit of profit....
...so it's all worth it in the end, right?
So that's not the case for you now?
I have some news for you about life for the vast majority of people right now... Including yourself, most likely
You may feel like a cog in society, and at present, this is often not the case, compared to the situation which I talked about in a sibling comment[2].
If you disagree, try this: find a new job and put in your resignation and say that you're leaving effective immediately (or with a very short notice). Most likely, you'll have your manager and HR scrambling to get a proper handover, and sometimes followed with private comments berating you for your "lack of professionalism" and other comments like "why are you doing this to us" and "can we keep you for a little longer, please" (in violation of HR policy for the maximum notice period).
In an age where AI has replaced most humans, people would lose all leverage, and would be forced into zero-sum games (killing and maiming other people to steal their wealth) as they would not have any other mechanism to prove their worth and improve their standing in society.
I don't know about you but maybe the fact so many people determine their entire worth from their job is more problematic than inventing technology that will enable people to pursue more worthwhile objectives of their own choosing.
Traditionally, these challenges have come through challenges associated with a hunter-gatherer or an agricultural lifestyle, and in contemporary times, it is associated with a job.
There are other forms which can make feel people rewarded, such as through the excess consumption of food or social media activity, but since they do not have an associated mental or physical challenge, they only have negative worth and cannot be the basis for someone's self worth and value.
Maybe it's time we found other "forcing functions"?
Furthermore, it seems silly to me to think that all human intellectual endeavors will simply disappear. We will still need to drive science, build bigger and better AI systems, improve and repair the world, etc.
And your takeaway from this would be that that worker is an invaluable member of the company? They don't care about workers, they care about their bottom line. If they could easily replace workers, they wouldn't say anything and would just make sure you clear out your desk before you leave.
And not that I agree my comment goes against the guidelines, but let me be generous and ask this of GP: What autonomy over your life now would you lose if AI were to become as prevalent as you're implying?
The reason citizens like you and me are kept around is because we have typically provided some form of economic output, in the form of labor and taxes, to society, and indirectly to the wealthy and powerful class, which is billionaires and politicians.
If most citizens don't have anything to contribute in an economy, they can be eliminated entirely from society so that this class can win the zero-sum games in society, and have natural resources to themselves. Thus, in the best case, commoners will be kept around in government provided housing (with all the things that it implies), with strict limits on reproduction and a very small baseline of resources so as to give the appearance that the government wasn't killing its people, though death would still be incentivized in the form of lower standards of medical care, so that society's dead weight can be eliminated.
In the worst case, politicians and billionaires would simply join forces to eliminate aforementioned dead weight through mass genocide. There is no downside to this, except that the first country to do this may have to face criticism, but I suspect eventually all countries would move in that direction once a few have taken their first step and established precedent.
I have to say, it's more confronting to hear it come from someone else.
The interesting part will be the violent conflict inherent in this turn of events. Will it be so easy to mow down billions of people? Will a way be found to do the job without the natural resistance of empathetic humans (A virus? A very large drone swarm? Pin-sharp training of a traditional police force?). How fast is too risky to execute? How _slow_ is too risky to execute?
Most people on this planet would figuratively kill for that life. It's a crushing existence trying to survive on below $10000/year.
Based on how the US currently rolls out universal programs, UBI probably looks a lot more like the current Section 8 housing program. They’re gonna guarantee something minimally livable, not beachfront property.
However, what can you really complain about? Technological progress? We can't just decide to ignore AI. Just to play devil's advocate, if there really were clear signs of this impending distruction, there could be some sort of international agreement to halt progress. Realistically, this will never happen.
> All along, we are incrementally improving AI because it is an intellectual amusement
You're contradicting yourself here. Is it just 'intellectual amusement' if this technology is as disruptive as you claim?
Let me be more clear: individual programmers are improving AI for its intellectual amusement, but organizations use it for its disruptive powers. Two different groups of people, with a bit of an intersection.
MOreover:
> Just to play devil's advocate, if there really were clear signs of this impending distruction, there could be some sort of international agreement to halt progress. Realistically, this will never happen.
Of course not, are you joking? There are clear signs of climate destruction as well with CO2 levels rising. Did international agreements work there? Nope, no flattening in the CO2 curve yet.
We are fundamentally destructive species, who cannot see long-term problems if there is short-term gain. The only mechanism we have on a global scale to decide what to do is capitalistic motivations.
So if AI destroys us as a species and...
> There are clear signs of climate destruction as well with CO2 levels rising. Did international agreements work there? Nope, no flattening in the CO2 curve yet.
Seems like we're in good shape?
If the end game is destroying ourselves, looks like our long term "problems" are solved.
That's clearer, but I still take issue with it. You can say the same about any software project or maybe even most work in general. As software engineers (I assume that's your profession too), we automate things that could have kept hundreds of people employed. There isn't that much different with AI - as long as there is money spent on the problem, there will be people willing to work on it, especially at the forefront of technolgy.
---
I agree with your second point. With that said, climate is way clearer distructive behavior, while also being more of a nuissance, a side effect of growth. AI has enormous potential and could, in the most optimistic outcomes, lead us to a utopia. Obviously, we all know that will not happen.
Also, the reason why AI progress will not be halted - we cannot allow our adversaries take the lead on this. It's really that simple.
Well I do agree with that. But I think that should also be re-examined and we should automate less...
https://ourworldindata.org/grapher/annual-co-emissions-by-re...
China has blown up emissions astronomically, though. To a lesser extent other Asian countries have as well.
I generally agree that international regulations controlling AI are unlikely to work, though, since it seems like it might be such a powerful and disruptive technology: if it doesn't stall, it could effectively be single-shot Prisoner's Dilemma, and when you have 193 players, someone's going to defect.
Personally though I think there are two possible outcomes:
1. Progress stalls, and it turns out getting from GPT-4-Turbo to better-than-human intelligence just doesn't pan out. LLMs are stuck as junior engineers for decades. If so, this is largely good for software engineers (and somewhat good for everyone, since it means we're more productive), but society doesn't change too much.
2. Progress doesn't stall, and we hit at least slightly-superhuman intelligence within the next decade. While this would obviously be a tough shift for most knowledge workers, especially depending on how quickly the shift happens, I also think this would likely bring about incredible medical advances, as well as incredible advances in robotics that reduce the cost of physical labor as well: meaning the price of goods drops enormously, and thanks to the medical advances we significantly increase either our lifespans, or at least the quality of our lives in old age, which seems quite positive. We'd need to figure out some sort of UBI system once the labor costs drop enough, but I think most people will be in favor of that, and also most stuff will just be really cheap at that point: ultimately just the cost of electricity (even "raw materials" are priced based on the cost of labor to get the materials, and the labor would be... the cost of electricity to run the robotics).
There are probably some in between scenarios, but TBH it's hard to see anything other than "stall" vs "takeoff" as being likely: either you never get past human intelligence (stall), or you do break through the wall, and then intelligence self-improves faster than before, up to some sort of information theory limit that I think is a lot higher than the average human is operating close to (consider just the variation in intelligence between individual human beings!).
Takeoff could also result in some sort of doomsday scenario, but the current LLMs haven't seemed to have the problems that the early doomers predicted, and so I think the like, humanity-enslaving or destroying outcomes are probably just not gonna happen.
That is why I used the word destructive instead of "having the tendency to destroy". All animals have that, but only we have actualized it. Hence we are destructive to a level that is unseen in other species.
It created this myopic view of labour relations where some workers think they will do better on their own against the powers-that-be, completely blind that they only have a tiny modicum of power while the elites look for any opportunity to strip that power away, be it through automation, collusion in cartel-like behaviour in the job market to push wages down, etc.
It's pretty obvious on HN how far this has come, in any topic discussing unions there will be quite a few of these illuded workers thinking they can do better on their own rather than show some solidarity to fellow workers, not too dissimilar than the American feeling of being just temporarily embarassed millionaires.
Remember, the social consequences of technological innovation are irrelevant in the Church of Modernity (no complainin' - what are you, some kinda heckin' Luddite!?). You will lose your way of life and you will be happy.
Obvious but also inevitable. Most of the discussions in this topic are short-sighted. We should look from a broader time perspective of human history. We're about to close the first quarter of 21th century with some initial advancements in AI. The second quarter will be all about transformation of the industries with the very possible introduction of AGI/ASI. This means the second half of the century must be a totally different world. With a totally different set of social constructs and different definitions for job, work, economy etc. I don't think we (humans, governments) are capable of changing the course of the history at the moment. We can only be cautious about the consequences.
People aren't developping AI because it's a hobby; people are doing it because it's an advancement in technology in the same way we developped electronics, computers, etc.
And those individual motivations add up to a collective force that is shaped via actions that persist with differential survivability based on capitalistic forces, which still have nothing to do with improving society. And they only reason why we have those is because at one time, free markets and trading were a good proxy for societal improvement on a small scale. On a large scale, they have become completely pathological.
Then it was the steam trains, and noone complained
Next it was the systems of electricity production, and noone complained
It's not. It's a tech demo. Try replacing your teammates with it...
AI at it's limit is practical guaranteed to destroy advance human society by concentrating mass power either to a single or small group of individuals, to itself or to everyone. Whichever happens the ultimate outcome is probably the same. Whether you open source or regulate, it really doesn't matter in the end. These are not solutions.
If a small group of individuals have access to AI they in effect become gods, that power will corrupt. If AI itself develops agency then it becomes our god. And if we all have the power of gods then it's just a matter of time before someone decides they want to use this power for destruction.
What we're building here is fundamentally destabilising and that's not even factoring in all the near-term risks of AI like mass job loses. And I think it's frankly disgusting how so many people dismiss this. I was checking out the VFX subreddit post the Sora announcement and many people there were literally suicidal at the prospect of their passion and economic value become obsolete in perhaps just a matter of years. It's heartbreaking.
And who even wants this future? We act like we have no choice the matter and maybe long-term that's true, but right now at the very least we have the chance to slow this rate of progress down and reduce near-term risk.
I know I sound like lunatic in all these AI doomer threads, but I continue to feel the need to normalise the obvious radical position that we need a UN agreement prohibiting the construction of AI data centers. And where data centers are built that exceed thresholds can and should be destroyed. As a global community we must also encourage anyone working in secret AI teams to become whistle blowers so that we can reduce the risk of governments or corporations building advance AI systems in secret.
And who knows, this might only buy as 3 or 4 decades, but at the very least we will give ourselves more time to improve our biological intelligence and equip ourselves with ways to deal with the risks AI presents.
The problem is that nearly half the population in America wants to give their freedom away and create a dictator to stick it to the "elites" so moving away from a concentration of power is probably not in the cards.
I don't disagree. That's why I'm concerned. I think what's certain is that if we want any chance at getting this right we must slow the exponential rate of progress.
We never complained about CAD systems replacing people who were drafting blueprints manually, not to mention all kinds of automation tools that replaced factory workers.
Why should AI coders be the same? If you can automate it, humans shouldn’t really be wasting their times on it.
There is nothing intellectual in pulling someone’s code from github to piece together some software by copy-pasting snippets.
Kentaro Toyama talks about technology as an amplifier for human intent.
Just as you could tighten a nut with your hands, you can tighten it so much more with a wrench.
Whether that nut is employed in a machine that will bring bounty for a village, or it's employed in a machine whose output by law belongs to its private owners is entirely up to the inhabitants.
Will the villagers band together and build their own machine?
Or will the authorities give the private creator of the machine exclusivity of the land the machine runs on and its output, leaving the villagers bereft?
Whatever state our world is in is a result of our peers decision, not "good" or "bad" technology. Humans will legislate: "good" and "bad" uses are up to them, and later on up to the executive and judicial system.
Pro-social (*not the same as socialist) cultures will share the wealth, others will hoard it, and you get the effects you dread: unemployment, poverty, etc.
Now, a better question is: is a technology too powerful or misunderstood that it might be misused, regardless of the creator's intent?
Aima, find a lucrative market for a new app Aifa, determine the cost building the new app Aiba, create the business model for the new app Aisa, create the use case functional spec for the new app Devin, create the new app
The ultimate goal of an organization (and society) is to stay alive, with minimum changes. It will evolve if it needs to, but it won't if it's not necessary.
Disrupting happens in a really really competitive enironment, and is only ONE of the different possible strategies to survive. Most of the time, disruption happens to the whole domain, not to the organization.
Others might be to "buy" the competitor, to build "regulatory barrier"...
This is a total reshaping of society at worst, and a huge transfer of wealth with almost no steps at best. How does anyone working on this justify causing unemployment on a mass scale while collecting a huge check? Is it just "progress gonna progress?" Do you really believe that justifies all the consequences? Is it, "if not me/us then someone else (worse)"? Then why aren't you giving control of these tools to the workers? Why are you profiting so handsomely?
This is nuts. Normally I don't go full communist rant, but a few hundred people (presumably some of whom read HN) are about to destroy our society without asking a single one of us, and with absolutely no regard for the consequences.
It is reshaping society in much the same way the industrial revolution did. Luddites resisted change, for precisely the same reason people aren't happy about AI, and it's now a derogatory label.
It's much better to adapt to a changing world but precisely what that entails is still up in the air.
Why does tech believe there should be no constraints whatsoever on what we build? Either our "change the world" pitches were completely inflated--what we do is meaningless dross--and we don't need to be regulated, or our mission statements were bang on and like any world-shaping thing we desperately need need to be regulated. You can't have it both ways.
I'm not against AI regulation on safety grounds but I'm not convinced it's a good idea to try to use regulation to preserve jobs. That feels like it would have much more damaging longer term consequences.
Not at all what I'm saying. What should they have done? Not trained on code they weren't explicitly licensed to do so, and only given access to workers.
The message of companies offering "productivity gains" (or outright replacement: "Hire Devin") to workers is "we're only interested in putting you out of work; how you pay your mortgage and get health care is your concern; call your congressperson". The fact that they're pushing hard to replace workers and flat out stating that their business models are illegal under current copyright law says everything about their offering: hey big business, if we put a carve out in copyright law your SWE headcount goes to zero; call your congressperson.
Hence my original--earnest--question: do you all who are working on AI honestly think the government will fix this?
Thankfully I believe this is largely just overblown hype. Silicon Valley has gotten way too high on it's own supply of bullshit and the AI craze is that being taken to the next level of insanity. Regardless, the tech industry is amoral at best. We are so obsessed with "can we" (or rather "can we convince a VC that we could") that we never take the time to stop to consider "should we".
Ultimately, watching tech find new and innovative ways to fool and to fleece people has, at times, almost made me feel shame that I've been part of this industry for almost 25 years now.
We are addicted to the idea of disruption and the monetary rewards that come with it. Generally we don't care if we sink the rest of the world in the process either.
Damn shame, that.
Agree that we don’t think enough about it.
People will adapt and find new things to do just like they did in the first, second and third industrial revolutions already.
Humans still need to do agricultural labour, the only difference is that machines mean less humans need to do that as a percentage of the population.
AI doesn't improve human labour productivity, but makes it obsolete in a similar way to how the combustion engine made the horse obsolete.
What exactly are you going to do of economic value when an AI can do everything the human mind can do?
EDIT:
> first, second and third industrial revolutions
These killed and maimed millions of people
Generative AI is still terrible at producing content people would actually pay to access. It’s hit the commissions market because the commissioner has some idea they want and someone who knows how to do the promos can make a lot of images quickly. Yet, Disney animation isn’t laying off anyone these images don’t suck but they’re also not good enough yet. Give it 40 years and who knows but that’s not tomorrow.
In terms of programming yes in very specific situations these things can improve productivity, but so can new libraries etc. It’s a long way from actually replacing programmers because what they are good at is what there’s examples that are close to it in the training data. Ask for a basic weather simulation using different atmospheric gasses and it’s not going to know what to do.
In such an economy, it may be that we can become communists! One of Marx's ideas was that technology will liberate man from labor. In actual historical Marxist societies, slave labor and coercion were widespread. But, in the future, if all labor is done by robots, far-left, restributionist policies may be viable because there's no humans being exploited (except, ironically, the humans who invented AI -- the people you're criticizing). In such a communist utopia no one will have to work, people will just have hobbies and hang out.
Are we heading towards this sci-fi scenario? I doubt it. I'm just trying to make two points: 1. when we imagine a future world with AI, there's no reason to imagine that it's a bad one and 2. it could be the case that "from each according to his ability, to each according to his needs" becomes a viable credo in an imaginary post-AI world.
This right here, this is 90% why.
With 10% of "what this button do?"