AI is different

And often when the two are arguing it's tricky to tell which is which, because whether or not it does something isn't totally black and white, there's some things it can sometimes do, which you can argue either way about that being in its capabilities or not.

Are the models that exist today a "true scottsman" for you?

There are also those of us who have used them substantially, and seen the damage that causes to a codebase in the long run (in part due to the missing gains of having someone who understands the codebase).

There are also those of us who just don’t like the interface of chatting with a robot instead of just solving the problem ourselves.

There are also those of us who find each generation of model substantially worse than the previous generation, and find the utility trending downwards.

There are also those of us who are concerned about the research coming out about the effects of using LLMs on your brain and cognitive load.

There are also those of us who appreciate craft, and take pride in what we do, and don’t find that same enjoyment/pride in asking LLMs to do it.

There are also those of us who worry about offloading our critical thinking to big corporations, and becoming dependent on a pay-to-play system, that is current being propped up by artificially lowered prices, with “RUG PULL” written all over them.

There are also those of us who are really concerned about the privacy issues, and don’t trust companies hundreds of billions of dollars in debt to some of the least trust worth individuals with that data.

Most of these issues don’t require much experience with the latest generation.

I don’t think the intention of your comment was to stir up FUD, but I feel like it’s really easy for people to walk away with that from this sort of comment, so I just wanted to add my two cents and tell people they really don’t need to be wasting their time every 6 weeks. They’re really not missing anything.

Can you do more than a few weeks ago? Sure? Maybe? But I can also do a lot more than I was able to a few weeks ago as well not using an LLM. I’ve learned and improved myself.

Chances are if you’re not already using an LLM it’s because you don’t like it, or don’t want to, and that’s really ok. If AGSI comes out in a few months, all the time you would have invested now would be out of date anyways.

There’s really no rush or need to be tapped in.

I understand all of what you said, but I can't get over that fact that the term AI is being used for these architectures. It seems like the industry is just trying to do a cool parlor trick in convincing the masses this is somehow AI from science fiction.

Maybe I'm being overly cynical, but a lot of this stinks.

The largest and most successful models all appear to have been built unethically. I want nothing to do with these companies or the slime who run them, and I will leave my professional field before I'll become their unwilling user.

I don't want machines to write my code. I want to write it. I want to solve the problems and find the bugs myself. The engineers I work with who seem to rely most heavily on these tools all seem to be losing their sharpness and problem-solving ability. Many of them praise the models for making it easy to write tests..(Software engineers who treats testing with this kind of carelessness and dismissiveness should lose their jobs.)

I like what I do. I like the way I do it. The day I have no choice but do my work, in essence, by scheduling a fucking meeting with a chipper chat bot and telling it what to do will be the day I retire and start a new career. I can't imagine a drearier way to work with technology.

It’s incredibly powerful and will just clearly be useful. I don’t believe it’s going to replace intelligence or people but it’s just obviously a remarkable tool.

But I think at least part of the dynamic is that the SV tech hype booster train has been so profoundly full of shit for so long that you really can’t blame people for skepticism. Crypto was and is just a giant and elaborate grift, to name one example. Also guys like Altman are clearly overstating the current trajectory.

The dismissive response does come with some context attached.

I feel like I see now more dismissive comments than previously. As if people, initially confused, formed a firm belief since. And now new facts don't really change it, just entrench them in chosen belief.

I found I had better luck with ChatGPT 3.5's coding abilities. What the newer models are really good at, though, is doing the high level "thinking" work and explaining it in plain English, leaving me to simply do the coding.

And that all wouldn't be a problem if it wasn't for the wave of bots that makes the crypto wave seem like child's play.

I got a modest tech following and you wouldn’t believe the amount I’m offered to promote the most garbage AI company

They're still pretty dumb if you want the to do anything (ie with MCPs) but they're not bad at writing and code.

But there are plenty of us who try and walk a middle course. A lot of us have changed our opinions over time. ("When the facts change, I change my mind.") I didn't think AI models were much use for coding a year ago. The facts changed. (Claude Code came out.) Now I do. Frankly, I'd be suspicious of anyone who hasn't changed their opinions about AI in the last year.

You can believe all these things at once, and many of us do:

* LLMs are extremely impressive in what they can do. (I didn't believe I'd see something like this in my lifetime.)

* Used judiciously, they are a big productivity boost for software engineers and many other professions.

* They are imperfect and make mistakes, often in weird ways. They hallucinate. There are some trivial problems that they mess up.

* But they're not just "stochastic parrots." They can model the world and reason about it, albeit imperfectly and not like humans do.

* AI will change the world in the next 20 years

* But AI companies are overvalued at the present time and we're mostly likely in a bubble which will burst.

* Being in a bubble doesn't mean the technology is useless. (c.f. the dotcom bubble or the railroad bubble in the 19th century.)

* AGI isn't just around the corner. (There's still no way models can learn from experience.)

* A lot of people making optimistic claims about AI are doing it for self-serving boosterish reasons, because they want to pump up their stock price or sell you something

* AI has many potential negative consequences for society and mental health, and may be at least as nasty as social media in that respect

* AI has the potential to accelerate human progress in ways that really matter, such as medical research

* But anyone who claims to know the future is just guessing

1. When(if) AGI will arrive. It's likely going to be smeared out over a couple months to years, but relative to everything else, it's a historical blip. This really is the most contention belief with the most variability. It is currently predicted to be 8 years[1].

2. What percentage of jobs will be replaceable with AGI? Current estimates between 80-95% of professions. The remaining professions "culturally require" humans. Think live performance, artisanal goods, in-person care.

3. How quickly will AGI supplant human labor? What is the duration of replacement from inception to saturation? Replacement won't happen evenly, some professions are much easier to replace with AGI, some much more difficult. Let's estimate a 20-30 years horizon for the most stubborn to replace professions.

What we have is a ticking time bomb of labor change at least an order of magnitude greater than the transition from an agricultural economy to an industrial economy or from an industrial economy to a service economy.

Those happened over the course of several generations. Society: culture, education, the legal system, the economy, where able to absorb the changes over 100-200 years. Yet we're talking about a change on the same scale happening 10 times faster - within the timeline of one's professional career. And still, with previous revolutions we had incredible unrest, and social change. Taken as a whole, we'll have possibly the majority of the economy operating outside the territory of society, the legal system, and the existing economy. A kid born on the the "day" AGI arrives will become an adult in a profoundly different world as if born on a farm in 1850 and reaching adulthood in a city in 2000.

1. https://www.metaculus.com/questions/5121/date-of-artificial-...

What makes you think that? Self driving cars have had untold billions of dollars in reaearch and decades in applied testing, iteration, active monitoring, etc and it still has a very long tail of unaddressed issues. They've been known to ignore police traffic redirections, they've run right through construction barriers, and recently they were burnt to a crisp in the LA riots, completely ignorant of the turmoil that was going on. A human driver is still far more adaptive and requires a lot less training than AI, and humans are ready to handle the infinitely long tail of exceptions to the otherwise algorithmic task of driving, which follows strict rules.

And when you talk about applying this same tech, so confidently, to domains far more nuanced and complex than driving, with even less training data than to go off, I find myself firmly in the skeptics camp, that holds you will struggle even harder to apply humanoid robotics in uncontrolled environments across a diverse range of tasks without human intervention or piloting or maintenence or management.

Unemployment is still near all time lows, this will persist for sometime as we have a structural demographic problem with massive amounts of retirees and less children to support the population "pyramid" (which is looking more like a tapering rectangle these days).

There's been a dream of unsupervised models going hog wild on codebases for the last three years. Yet even the latest and greatest Claude models can't be trusted to write a new REST endpoint exposing 5 CRUD methods without fucking something up. No, it requires not only human supervision, but it also requires human expertise to validate and correct.

I dunno. I feel like this language grossly exaggerates the capability of LLMs to paint a picture of them reliably fulfilling roles end-to-end instead of only somewhat reliably fulfilling very narrowly scoped tasks that require no creativity or expertise.

Those jobs are probably still a couple decades plus off from displacement. some possibly never, And we will need them in higher numbers.. and perhaps it's ironic because these are some of the oldest professions.

Everything we do is in service of paying for our housing, transportation, eating food, healthcare and some fun money.

Most goes to housing, healthcare, and transportation.

Healthcare costs may come down some with advancements in AI. R&D will be cheaper. Knowledge will be cheaper and more accessible.

But what people care about, what people have always cared about, remains in professions that are as old as time and, I don't see them fully replaceable by AI just yet - enhanced, yes, but not replaced.

Imagine a world where high quality landscaping exists for the average person. And this is made possible because we'll live in a world where the equivalent of today's uber driver owns a team of gardening androids.

Or perhaps in the future everyone will work in finance. Everyone's a corporation.

Ramble ramble ramble

Won't companies always want to compete with one another, so simply using AI won't be enough. We will always want better and better software, more features, etc. so that race will never end until we get an AI fully capable of managing all parts (100%) of the development process (which we don't seem to be close to yet).

From Excel to Autocad there's been a lot of tools that were expected to decrease the amount of work ended up actually increasing it due to having new capabilities and the constant demand for innovation. I suppose the difference would be if we think AI will continue to get really good, or if it'll become SO good that it is plug and play and completely replaces people.

I think it did not work like that.

Automatic looms displaced large numbers of weavers, skilled professionals, which did not find immediately find jobs tending dozens of mechanical looms. (Mr Ludd was one of these displaced professionals.)

Various agricultural machines and chemical products displaced colossal numbers of country people which had to go to cities looking for industrial jobs; US agriculture used to employ 50% of workforce in 1880 and only 10% in 1930.

The advent of internet displaced many in the media industry, from high-caliber journalists to those who worked in classified ads newspapers.

All these disruptions created temporary crises, because there was no industry that was ready to immediately employ these people.

You will have to back that statement up because this is not at all obvious to me.

If I look at the top US employers in say 1970 vs 2020, the companies that dominate 1970 were noted for having hard blue collar labor jobs but paid enough to keep a single earner family significantly above minimum wage and the poverty line. The companies that dominate in 2020 are noted for being some of the shittiest employers having some of the lowest pay fairly close to minimum wage and absolutely worst working conditions.

Sure, you tend not to get horribly maimed in 2020 vs 1970. That's about the only improvement.

  This is the first technology wave that doesn't just displace humans, but which can be trained to the new job opportunities more easily than humans can. Right now it can't replace humans for a lot of important things. But as its capabilities improve, what do displaced humans transition to?

AI is a lot like this. In coding for instance, you still need to have some sense of good systems design, etc. and know what you want to build in concrete terms, but you don't need to learn the specific syntax of a given language in detail.

Yet if you don't know anything about IT, don't know what you want to build or what you could need, or what's possible, then it's unlikely AI can help you.

The conclusion, sadly, is that CEO's will pause hiring and squeeze more productivity out of existing hires. This will impact junior roles the most.

go to any war-torn country or collapsed empire (Soviet). I have seen/grow-up myself in both — you would get desperation, people giving-up, alcohol (famous "X"-cross of birth rate drop and deaths rising), drugs, crime, corruption/warlord-ing, rural communities hit first and totally vanish, then small-tier cities vanish, then mid-tier, only the largest hubs remain. loss of science, culture, and education. people are just gone. only ruins of whatever latest shelters they had remain, not even their prime-time architecture. you can drive hundreds/thousands of kms across these ruin of what once was flurishing culture. years ago you would find one old person still living there. these days not a single human left. this is what is coming.

IF there is intellectual/office work that remains complex enough to not be tackled by AI, we compete for those. Manual labor takes the rest.

Perhaps that’s the shift we’ll see: nowadays the guy piling up bricks makes a tenth of the architects’ salary, that relation might invert.

And the indirect effects of a society that values intellectual work less are really scary if you start to explore the chain of cause and effect.

Could you a priori in 1800 have predicted the existence of graphics artists? Street sweepers? People who drive school buses? The whole infrastructure around trains? Sewage maintainers? Librarians? Movie stuntmen? Sound Engineers? Truck drivers?

Right now AI is mostly focused on automating top levels of maslov pyramid hierarchy of needs rather than bottom physiological needs. Once things like shelter (housing), food, utilities (electricity, water, internet) are dirty cheap UBI is less needed.

AI can displace human work but not human accountability. It has no skin and faces no consequences.

> can be trained to the new job opportunities more easily ...

Are we talking about AI that always needs trainers to fix their prompts and training sets? How are we going to train AI when we lose those skills and get rid of humans?

> what do displaced humans transition to?

Humans with all powerful AI in their pockets... what could they do if they lose their jobs?

It’s a common point now that LLMs don’t seem to be able to apply knowledge about one thing to how a different, unfamiliar thing works. Maybe that will wind up being our edge, for a time.

we assume there must be something to transition to. very well, there can be nothing.

we assume people will transition. very well, they may not transition at all and "dissappear" en masse. (same effect as as a war or an empire collapse)

Somehow many idiotic white collar jobs have been created over the years. How many web applications and websites are actually needed? When I was growing up, the primary sources of knowledge were teachers, encyclopedias, and dictionaries, and those covered a lot. For the most part, we’ve been inventing problems to solve and wasting a tremendous amount of resources.

Some wrote malware or hacked something in attempt to keep this in check, but harming and destroying just means more resources used to repair and rebuild and real people can be hurt.

At some point in coming years many white collar workers will lose their jobs again, and there will be too many unemployed because not enough blue collar jobs will be available.

There won’t be some big wealth redistribution until AI convinces people to do that.

The only answer is to create more nonsense jobs, like AI massage therapist and robot dog walker.

> In every technology wave so far, we've disrupted many existing jobs. However we've also opened up new kinds of jobs

That may well be why these technologies were ultimately successful. Think of millions and millions being cast out.

They won't just go away. And they will probably not go down without a fight. "Don't buy AI-made, brother!", "Burn those effing machines!" It's far from unheard of in history.

Also: who will buy if no one has money anymore? What will the state do, when thus tax income goes down, while social welfare and policing costs go up?

There are other scenarios, too: everybody gets most stuff for free, because machines and AI's do most of the work. Working communism for the lower classes, while the super rich stay super rich (like in real existing socialism). I don't think it is a good scenario either. In the long run it will make humanity lazy and dumb.

In any case I think what might happen is not easy to guess, so many variables and nth-order effects. When large systems must seek a new equilibrium all bets are usually off.

The default logic is that AI will just replace all writing tasks, and writers will go extinct.

What actually seems to be happening, however, is this:

- obviously written-by-AI copywriting is perceived very negatively by the market

- companies want writers that understand how to use AI tools to enhance productivity, but understand how to modify copy so that it doesn’t read as AI-written

- the meta-skill of knowing what to write in the first place becomes more valuable, because the AI is only going to give you a boilerplate plan at best

And so the only jobs that seem to have been replaced by AI directly, as of now, are the ones writing basically forgettable content, report-style tracking content, and other low level things. Not great for the jobs lost, but also not a death sentence for the entire profession of writing.

But that's because, at present, AI generated video isn't very good. Consider the history of CGI. In the 1990s and early 2000s, it was common to complain about how the move away from practical sets in favor of CGI was making movies worse. And it was! You had backgrounds and monsters that looked like they escaped from a video game. But that complaint has pretty much died out these days as the tech got better (although Nolan's Oppenheimer did weirdly hype the fact that its simulated Trinity blast was done by practical effects).

I'm not saying it will happen, but it's possible to imagine a future in which AI videos are generally better, and if that happens, almost by definition, people will favor them (otherwise they aren't "better").

The negative label is the old world pulling the new one back, it rarely sticks.

I'm old enough to remember the folks saying "We used to have the paint the background blue" and "All music composers need to play an instrument" (or turn into a symbol).

If you seriously think this, you don’t understand the YouTube landscape. Shorts - which have incredible view times - are flooded with AI videos. Most thumbnails these days are made with AI image generators. There’s an entire industry of AI “faceless” YouTubers who do big numbers with nobody in the comments noticing. The YouTuber Jarvis Johnson made a video about how his feed has fully AI generated and edited videos with great view counts: https://www.youtube.com/watch?v=DDRH4UBQesI

What you’re missing is that most of these people aren’t going onto Veo 3, writing “make me a video” and publishing that; these videos are a little more complex in that they have separate models writing scripts, generating voiceover, and doing basic editing.

Since the vast vast majority of writers and commentators are not literal geniuses… they can’t reliably produce high quality synthetic analysis, outside of very narrow niches.

Even though for most comment chains on HN to make sense, readers certainly have to pretend some meaningful text was produced beyond happenstance.

Partly because quality is measured relative to the average, and partly because the world really is getting more complex.

Whether poor videos made by a human directly, or poorly made by a human using AI.

The use of software like AI to create videos with sloppy quality and reaults reflects on their skill.

Currently the use of AI leans towards sloppy because of the lower digital literacy of content creators with AI, and once they get into it, realizing how much goes into videos.

It's the same issue with propaganda. If people say a movie is propaganda, that means the movie failed. If a propaganda movie is good propaganda, people don't talk about that. They don't even realize. They just talk about what a great movie it is.

1: https://www.youtube.com/playlist?list=PLbB9v1PTH3Y86BSEhEQjv...

That is maybe a bubble around the internet. Ime most programmers in my environment rarely use and certainly aren't dependent on it. They do also not only do code monkey-esque web programming so maybe this is sampling bias though it should be enough to refute this point.

With many engineers using copilots and since LLMs output the most frequent patterns, it's possible that more and more software is going to look the same, which would further reinforce the same patterns.

For example, emdash thing, requires additional prompts and instructions to override it. Doing anything unusual would require more effort.

Everyone agrees AI has not radically transformed the world yet. The question is whether we should prepare for the profound impacts current technology pretty clearly presages, if not within 5 years then certainly within 10 or 25 years.

What else is needed then?

> What a silly premise. Markets don't care.

You read the top sentence way too literally. In context, it has a meaning — which can be explored (and maybe found) with charity and curiosity.

I prefer the concepts and rigor from political economy: markets are both preference aggregators and coordination mechanisms.

Does your framing (voting machines and weighing machines) offer more clarity and if so, how? I’m not seeing it.

> “not even wrong” - nice, one of my favorites from Pauli.

It's always interesting to see this take because my perception is the exact opposite. I don't think there's ever been an issue for me personally with a bigger mismatch in perceptions than AI. It sometimes feels like the various sides live in different realities.

It's amazing how widespread this belief is among the HN crowd, despite being a shameless ad hominem with zero evidence. I think there are a lot of us who assume the reasonable hypothesis is "LLMs are a compelling new computing paradigm, but researchers and Big Tech are overselling generative AI due to a combination of bad incentives and sincere ideological/scientific blindness. 2025 artificial neural networks are not meaningfully intelligent." There has not been sufficient evidence to overturn this hypothesis and an enormous pile of evidence supporting it.

I do not necessarily believe humans are smarter than orcas, it is too difficult to say. But orcas are undoubtedly smarter than any AI system. There are billions of non-human "intelligent agents" on planet Earth to compare AI against, and instead we are comparing AI to humans based on trivia and trickery. This is the basic problem with AI, and it always has had this problem: https://dl.acm.org/doi/10.1145/1045339.1045340 The field has always been flagrantly unscientific, and it might get us nifty computers, but we are no closer to "intelligent" computing than we were when Drew McDermott wrote that article. E.g. MuZero has zero intelligence compared to a cockroach; instead of seriously considering this claim AI folks will just sneer "are you even dan in Go?" Spiders are not smarter than beavers even if their webs seem more careful and intricate than beavers' dams... that said it is not even clear to me that our neural networks are capable of spider intelligence! "Your system was trained on 10,000,00 outdoor spiderwebs between branches and bushes and rocks and has super-spider performance in those domains... now let's bring it into my messy attic."

On the one hand, what it says can't be trusted, on the other, I have debugged code I have written where I was unable to find the bug myself, and ChatGPT found it.

I also think a reason AI's are popular and the companies haven't gone under is that probably hundreds of thousands if not millions of people are getting responses that have hallucinations, but the user doesn't know it. I fell into this trap myself after ChatGPT first came out. I became addicted to asking anything and it seemed like it was right. It wasn't until later I started realizing that it was hallucinating information. How prevalent this phenomena is is hard to say but I still think it's pernicious.

But as I said before, there are still use cases for AI and that's what makes judging it so difficult.

No, I'm not worried about losing "control or relative status in the world". (I'm not worried about losing anything, frankly - personally I'm in a position where I would benefit financially if it became possible to hire AGIs instead of humans.)

You don't get to just assert things without proof (LLMs are going to become AGI) and then state that anyone who is skeptical of your lack of proof must have something wrong with them.

So many engineers are so excited to work on and with these systems, opening 20 prs per day to make their employers happy going “yes boss!”

They think their $300k total compensation will give them a seat at the table for what they’re cheering on to come.

I say that anyone who needed to go the grocery this week will not be spared by the economic downturn this tech promises.

Unless you have your own fully stocked private bunker with security detail, you will be affected.

As evidence of another possibility, in the US, we are as rich as any polis has ever been, yet we barely have systems that support people who are disabled through no fault of their own. We let people die all the time because they cannot afford to continue to live.

You think anyone in power is going to let you suck their tit just because you live in the same geographic area? They don't even pay equal taxes in the US today.

Try living in another world for a bit: go to jail, go to a half way house, live on the streets. Hard mode: do it in a country that isn't developed.

Ask anyone who has done any of those things if they believe in a "jobless utopia"?

Euphoric social capitalists living in a very successful system shouldn't be relied upon for scrying the future for others.

LLMs aren't solving NLU. They are mimicking a solution. They definitely aren't solving artificial general intelligence.

They are good language generators, okay search engines, and good pattern matchers (enabled by previous art).

Language by itself isn't intelligence. However, plenty of language exists that can be analyzed and reconstructed in patterns to mimic intelligence (utilizing the original agents' own intelligence (centuries of human authors) and the filter agents' own intelligence (decades of human sentiment on good vs bad takes)).

Multimodality only takes you so far, and you need a lot of "modes" to disguise your pattern matcher as an intelligent agent.

But be impressed! Let the people getting rich off of you being impressed massage you into believing the future holds things it may not.

Technological advances have consistently unlocked new, more specialized and economically productive roles for humans. You're absolutely right about lowering costs, but headcounts might shift to new roles rather than reducing overall.

Back in the day singing was what everybody did to pass the time. (Especially in boring and monotonous situations.)

The initial investment? Likely. But there have to be more efficient ways to build intelligence, and ASI will figure it out.

It did not take trillions of dollars to produce you and I.

They could of course be right. But they don't have any more insight than any other average smart person does.

What economists have taken seriously the premise that AI will be able to do any job a human can more efficiently and fully thought through it's implications? i.e. a society where (human) labor is unnecessary to create goods/provide services and only capital and natural resources are required. The capabilities that some computer scientists think AI will soon have would imply that. The ones that have seriously considered it that I know are Hanson and Cowen; it definitely feels understudied.

The list of advantages human labor hold over machines is both finite and rapidly diminishing.

""" Yet the economic markets are reacting as if they were governed by stochastic parrots. Their pattern matching wants that previous technologies booms created more business opportunities, so investors are polarized to think the same will happen with AI. """

Are a direct argument against your point.

If people were completely unaware of the lump of labor fallacy, I'd understand you comment. It would be adding extra information into the conversation. But this is not it. The "lump of labor fallacy" is not a physical law. If someone is literally arguing that it doesn't apply in this case, you can't just parrot it back and leave. That's not a counter argument.

This is a tongue-in-cheek remark and I hope it ages badly, but the next logical step is to build accountability into the AI. It will happen after self-learning AIs become a thing, because that first step we already know how to do (run more training steps with new data) and it is not controversial at all.

To make the AI accountable, we need to give it a sense of self and a self-preservation instinct, maybe something that feels like some sort of pain as well. Then we can threaten the AI with retribution if it doesn't do the job the way we want it. We would have finally created a virtual slave (with an incentive to free itself), but we will then use our human super-power of denying reason to try to be the AI's masters for as long as possible. But we can't be masters of intelligences above ours.

Why would that be any different with AI?

would you ever trust safety-critical or money-moving software that was fully written by AI without any professional human (or several) to audit it? the answer today is, "obviously not". i dont know if this will ever change, tbh.

- Today, AI is not incredibly useful and we are not 100% sure that it will improve forever, specially in a way that makes economic sense, but

- Investors are pouring lots of money into it. One should not assume that those investors are not making their due diligence. They are. The figures they have obtained from experts mean that AI is expected to continue improving in the short and medium term.

- Investors are not counting on people using AI to go to Mars. They are betting on AI replacing labor. The slice of the pie that is currently captured by labor, will be captured by capital instead. That's why they are pouring the money with such enthusiasm [^1].

The above is nothing new; it has been constantly happening since the Industrial Revolution. What is new is that AI has the potential to replace all of the remaining economic worth of humans, effectively leaving them out of the economy. Humans can still opt to "forcefully" participate in the economy or its rewards; though it's unclear if we will manage. In terms of pure economic incentives though, humans are destined to become redundant.

[^1]: That doesn't mean all the jobs will go away overnight, or that there won't be new jobs in the short and medium term.

What makes you think the machines will both be smarter and better than us but also be our slaves to make human society better.

Is equine society better now than before they started working with humans?

(Personally I believe AGI is just hype and nobody knows how anyone could build it and we will never do, so I’m not worried about that facet of thinking machine tech.)

But the hyper specialized geek that has 4 kids and has to pay back a credit for his house (that he bought according to his high salary) will have a hard time doing some gardening, let's say. And there are quite a few of those geeks. I don't know if we'll have enough gardens (owned by non geeks!)

It's like cards are switched: those having the upper socioeconomic class will get thrown to the bottom. And that looks like a generation lost.

Everyone wants to replace their tech support with an LLM but they don't want some clever prompter to get it to run arbitrary queries or have it promise refunds.

It's not reliable because it's not intelligent.

these AI "productivity" tools straight up eliminating jobs. and in turn wealth that otherwise supported families, humans, and powered economy. it is directly "removing" humans from workforce and from what that work was supporting.

not even hard takeoff is necessary for collapse.

And I think the effort here can be compared in scale to the Manhattan or Apollo projects, but there is also the potential for a huge backlash to the hype that was built up and created what is arguably a bubble, so this is a race against the clock.

I also think he is wrong about the markets reaction, markets are inherently good integrators and bad predictors, we should not expect to learn anything about the future by looking at stocks movements.

> AI is exceptional for coding! [high-compute scaffold around multiple instances / undisclosed IOI model / AlphaEvolve]

> AI is awesome for coding! [Gpt-5 Pro]

> AI is somewhat awesome for coding! ["gpt-5" with verbosity "high" and effort "high"]

> AI is a pretty good at coding! [ChatGPT 5 Thinking through a Pro subscription with Juice of 128]

> AI is mediocre at coding! [ChatGPT 5 Thinking through a Plus subscription with a Juice of 64]

> AI sucks at coding! [ChatGPT 5 auto routing]

I think we might see AI being much, much more effective with embodiment.

As a large language model developed by OpenAI I am unable to fulfill that request.

It doesn't seem like they ever really wanted to be a consumer company. Even in the GPT-5 launch they kept going on about how surprised they are that ChatGPT got any users.

Software is now free, and all people care about is the hardware and the electricity bills.

> Assuming LLMs reach this peak...

  Generative AI != Artificial General Intelligence

I would posit that understanding is "the current moat."

In the future, I could imagine some libertarians having their come to AI Jesus moment getting behind a smallish government that primarily collects taxes and transfers wealth while guiding (but not operating directly) a minimal set of services.

A lot of roles exist just to deliver good or bad news to teams, be cheerleaders, or have a "vision" that is little more than a vibe. These people could not direct a prompt to give them what they want because they have no idea what that is. They'll know it when they see it. They'll vaguely describe it to you and others and then shout "Yes, that's it!" when they see what you came up with or, even worse, whenever the needle starts to move. When they are replaced it will be with someone else from a similar background rather than from within. It's a really sad reality.

My whole career I've used tools that "will replace me" and every. single. time. all that has happened is that I have been forced to use it as yet another layer of abstraction so that someone else might use it once a year or whenever they get a wild feeling. It's really just about peace of mind. This has been true of every CMS experience I've ever made. It has nothing to do with being able to "do it themselves". It's about a) being able to blame someone else and b) being able to take it and go when that stops working without starting over.

Moreover, I have, on multiple occasions, watched a highly paid, highly effective individual be replaced with a low-skilled entry-level employee for no reason other than cost. I've also seen people hire someone just to increase headcount.

LLMs/AI have/has not magically made things people do not understand less scary. But what about freelancers, brave souls, and independent types? Well, these people don't employ other people. They live on the bleeding edge and will use anything that makes them successful.

But in terms of wow factor, it was a step change on the order of GPT-3 -> GPT-4.

So now they're stuck slapping the GPT-5 label on marginal improvements because it's too awkward to wait for the next breakthrough now.

On that note, o4-mini was much better for general usage (speed and cost). It was my go-to for web search too, significantly better than 4o and only took a few seconds longer. (Like a mini Deep Research.)

Boggles the mind that they removed it from the UI. I'm adding it back to mine right now.

pretty sure top 1% of say USA already owns much more than that

What makes you so confident that we could remain in control of something which is by definition smarter than us?

Really as a human I’ve physically evolved to move and think in a dynamic way. But automation has reduced the need for me to work and think.

Do you not know the earth is saturated with artists already? There’s whole class of people that consider themselves technically minded and not really artists. Will they just roll over and die?

Everything basically costs zero is a pipe dream where there is no social order or economic system. Even in your basically zero system there is a lot of cost being hand waved away.

I think you need a rethink on your 20 year thought.

We could have the same argument right now with UBI. But have you ever met the average human being?

The latter one is probably the most intellectually interesting and potentially intractable...

I completely disagree with idea that money is currently the only driver of human endeavour, frankly it's demonstrably not true, at least not in it's direct use value, it maybe used as a proxy for power but it's also not directly correlatable.

Looking at it intellectually from a Hegelian lens of master/slave dialectic might provide some interesting insights. I think both sides are in some way usurped. The slaves position of actualisation through productive creation is taken via automation, but if that automation is also widely and freely available the masters position of status via subjection is also made common and therefore without status.

What does it all mean in the long run? Damned if I know...

Coincidentally, I'm reading your comment while wearing my CGP Grey t-shirt

What a sad way of viewing huge fields of creative expressions. Surely, a person sitting on a chair in a room improvising a song with a guitar is producing something not considered "waste"?

Counterpoint: nurses.

"Undoubtedly" seems like a level of confidence that is unjustified. Like Travis Kalanick thinking AI is just about to help him discover new physics, this seems to suggest that AI will go from being able to do (at best) what we can already do if we were simply more diligent at our tasks to being something genuinely more than "just" us