This is where we're headed: https://sibylline.dev/articles/2025-10-12-ai-is-too-big-to-f...
> For you AI skeptics, this is going to be rough to hear, but given the suffering that will occur if AI fails, I believe we have a moral imperative to make sure it succeeds.
What if, and hear me out here, the prediction is that it will fail because it can't succeed. It hasn't proven it is capable of producing the results that would justify itself. That there is some value doesn't mean that it will develop that capability. We have nothing to say that it will.
It might be possible that it can't succeed, but we don't know that and the evidence in terms of what happens if it fails is pretty compelling, so I think morality dictates we have to at least try, at least until we have more information or another path emerges.
What's doubly duplicitous is that even if LLMs achieve general intelligence, the whole idea is to enrich a tiny fraction of humanity that happens to be shareholders.
EDIT: trying to address the Roko part..I'm assuming once AGI is achieved.. the AGI doesn't need more compute to increase its intelligence beyond those of an average activist employee (I can assure you that in OpenAI there are such employees, and they know to shut up for now)
the antisocial part: it's already happening. What can you do about that.
As a thought experiment, say you were the CEO/board member of a company that's told your new platform is choosing public benefit over profits. What would you do? Now filter down the decisions going down the heirarchy, considering job security and a general preference for earning bonuses.
For all the discussions around "alignment" - any AI that's not aligned with increased profits will be unplugged posthaste, all other considerations are secondary.
The first example that comes to mind is OpenAI's Sebastian Bubeck (& others) giving underwhelming examples to illustrate that GPT has surpassed human mathematicians. Or earlier, when SamA says that he has seen things you wouldn't believe, whereas the top replicant in a proper sci-fi will at least be specific about the I-Beams.
Another parallel which you would be familiar with is nuclear power. Humans can plainly see it's an unworldly tech, but I'd say the value extracted so far has been a net negative-- mainly because our collective engineering chops are just too.. profane. And SamA/Musk/Thiel/Luckey just don't seem as wise as Rickover (who is the Rickover of our age? I think the Rickover pipeline is broken, tbh)
From my vantage point, I agree with you: China sees AI as less important as solar, so charitably, I'd say that Thiel's contrarian strategy is actually to get the Chinese so riled up they zuzwang themselves into providing the progress but keeping the whirlwind for themselves (so proving he has learnt the post-mimetic game from the atomics opening)
There could be another interesting post-mimetic line exploring how a hybrid chabuduo-"festina-lente" culture is required for developing supertech.. which.. only a Rickover can play
(I don't know if you're army or navy but there's this millenniums old cultural divide between a conservative army engineering culture -- that develops atomics -- and a progressive naval engineering culture -- that makes headway harnessing it. AI bulls smell like they are more aligned with land-forces
What does that look like?
But LLM is not an AI. Tesla self-driving tech is not an AI.
Even my dishwasher has AI according to the manufacturer!
Can you call it a success if washing machine has AI already?
I don't see how some kind of big breakthrough is going to happen with the current model designs. The superintelligence, if it will ever be created, will require a new breakthrough in model architecture. We've pretty much hit the limit of what is possible with current LLMs. The improvements are marginal at this point.
Secondly, hypothetically, the US achieves superintelligence, what is stopping China from doing the same in a month or two, for example?
Even if China achieves a big breakthrough first, it may benefit the rest of the world.
If you read the article carefully, I work hard to keep my priors and the priors of the people in question separate, as their actions may be rational under their priors, but irrational under other priors, and I feel it's worth understanding that nuance.
I'm curious where you got the writer "clinging to power and money desperately."
Also, to be fair, I envy Europe right now, but we can't take that path.
The game seems to be everyone else waiting for the R&D money to provide us amazing open source models, and then just run those locally.
My cynical take is that this is the US committing economic suicide, based on a misguided belief in something that'll never happen.
The new superpowers will be the EU, which was smart enough not to make the same gamble, and China, which will structurally survive it.
I also disagree with your conclusion of a moral imperative to make sure that AI succeeds. I believe it's the opposite. AI failing would finally create the long-needed revolutionary moment to throw off the shackles of the oligarchy that got us into this mess in the first place.
Not with how much pulling teeth is required to get them to invest in defense. I don't see how you can unironically make the claim that a written down investment would sink the ship that is the US economy.
A written down investment thats four times the size of the mortgage crisis.
Call me a buzzkill here, but my bet is you aren't gonna hit it big. While you can win at the casino, that requires a careful plan with contingency solutions, which AI simply does not have. Realistically speaking, it is indeed best to just not play this game; you are just digging a deeper hole.
As for defense, they are spending exactly as much as necessary at each point in time: just enough to keep credible US backing until 2025, and as much as they can without destroying their economy since. There is no good argument for an irresponsible spending spree, as the only powers that can realistically challenge the EU without triggering nuclear holocaust are the US and China anyways (Russia don't stand a chance).
We need political Aikido to hold this country together.
The competition between the US and China is pertinent to everyone.
Seeing the last tariffs and what China done about the rare earth minerals (and also the deal the US made with Ukraine for said minerals), the article might have a point that the super power will cripple each other to be the first with the super intelligence. And you also need money for it so tariffs.
(There are a thousand more problems, but none of them matter until that first one is overcome.)
I think if one really thinks about this statement, they'll find the opposite to be at least as possible.
(This is the punchline to a joke.)
Is it? I am pretty sure biology will solve good old "are viruses alive?" sooner than we agree on definition of intelligence. "Chinese Room" is at least 40 years old.
Practically speaking, the inherentness of intelligence doesn't really matter because both intelligent-looking entity and provably intelligent entity are capable for societal disruptions anyway. I partly dislike the Chinese Room argument for this reason; it facilitates useless discussions in most cases.
For the folks who lived though it; were the Expert Systems boosters as insufferable in the 80s as the LLM people are now about the path to machine intelligence?
ARPA would throw relatively large sums of money at you, but demand progress reports and a testable goal. Very little got rolled out based on hype. (Let's not talk about vehicle design.) If your project didn't show signs of working, or not enough signs of working, funding ended.
Anything which met goals and worked, we now think of as "automation" or "signal recognition" or "solvers", not "intelligent systems".
Tariffs aren’t there to pay for a race to superintelligence, they’re a lever that the authoritarian is pulling because it’s one of the most powerful levers the president is allowed to pull. It’s a narcissist’s toy to feel important and immediately impactful (and an obvious vehicle for insider trading).
If the present administration was interested in paying for a superintelligence race they wouldn’t have signed a law that increases the budget deficit.
They also wouldn’t be fucking with the “really smart foreign people to private US university, permanent residence, and corporate US employment” pipeline if they were interested in the superintelligence race.
I don’t underestimate their ability to do damage but calling them smart is generous.
Not even Peter Thiel, he’s one of the most over-hyped people in tech. Access to capital is not intelligence, and a lot of his capital comes from the equivalent of casino wins with PayPal and Facebook.
- this is war path funding
- this is geopolitics; and it’s arguably a rational and responsible play
- we should expect to see more nationalization
- whatever is at the other end of this seems like it will be extreme
And, the only way out is through
I believe China's open source focus is in part a play for legitimacy, and part a way to devalue our closed AI efforts. They want to be the dominant power not just by force but by mandate. They're also uniquely well positioned to take full advantage of AI proliferation as I mentioned, so in this case a rising tide raises some boats more than others.
They don't heavily advertise this definition because investors expect AGI to mean the computer from Her, and it's not gonna be that. They want to be able to tell investors without lying that they're on target for AGI in 3 years, and they're riding on pre-existing expectations.
Also thats 2028.. 2026 midterms look grim.
And I mean inevitably in the strongest way possible. This happens at basically every midterm with the opposition party picking up seats and republicans barely control Congress as it is.
> Chinese will not pull back on AI investment, and because of their advantages in robotics and energy infrastructure, they don't have to.
> The gap matters because AI's value capture depends on having the energy and hardware to deploy frontier models in the physical world.
I understand energy infrastructure because of power-hungry GPUs and ASICs, but I was wondering about the nexus between AI and robotics. I have my own thoughts on how they are linked, but I don't think my thoughts are the same as the author's and I would like to know more about them.
It's not tho. We've been at it for about 70 years now. Returns have been diminishing exponentially if you look at the amounts invested and we still have bumbling contraptions that are useful in very narrow and contrived use cases.
The whole hype is based on wishful/magical thinking. The booster arguments are invariably about some idea in their minds that has no correspondent in the real world.
Personally I do not think that we have seen the full battlefield impact of the last 10-20 years developments in microcontrollers, battery, power electronics, electromotors, MEMS sensors, imaging sensors, control firmware, computer vision and object detection. Capability per dollar has increased by a few orders of magnitude in each of these domains. We are seeing some hints of the potential impact now after Russia invaded Ukraine - low-cost drones are now killing and damaging just as much or more as existing high tech weaponry. These systems are crude, with limited autonomy, but it is already enough to wreck havoc. I do not think development of these systems have stopped, unfortunately.
The US indeed seems destined to fall behind due to decades of economic mismanagement under neoliberalism while China’s public investment has proved to be the wise choice. Yet this fact wounds the pride of many in the US, particularly its leaders, so it now lashes out in a way that hastens its decline.
The AI supremacy bet proposed is nuts. Prior to every societal transition the seeds of that transition were already present. We can see that already with AI: social media echo chambers, polarization, invading one’s own cities, oligarchy, mass surveillance.
So I think the author’s other proposed scenario is right - mass serfdom. The solution to that isn’t magical thinking but building mass solidarity. If you look at history and our present circumstances, our best bet to restore sanity to our society is mass strikes.
I think we are going to get there one way or another. Unfortunately things are probably going to have to get a lot more painful before enough people wake up to what we need to do.
Do you really prefer brutal serfdom to the AI supremacy scenario? From where I sit, people have mixed (trending positive) feelings about AI, and hard negative feelings about being in debt and living paycheck to paycheck. I'd like to understand your position here more.
What I found persuasive was the argument that this bubble could be worse than others due to the perceived geopolitical stakes by US leadership, plausibly leading to mass serfdom as our society implodes, based on the argument that we already have a version of serfdom today. I found that astute.
I do NOT think that scenario is favorable - it just seems like the most likely future. I hold that we should view our situation with clear eyes so that we can make the best decision from where we stand.
Thinking through this, what does that mean for how we should face our present moment? Eventually people will throw off their chains; our leadership is so incompetently invested in class war that the working class is going to continue to be squeezed and squeezed - until it pops. It’s a pattern familiar in history. The question in my mind is not if but when. And the sooner we get our collective act together the less painful it’s going to be. There’s some really bad stuff waiting for us round the bend.
What should we do? The economy depends on us the serfs and soon-to-be serfs. It’s the workers who can take control of it and can shut it down with mass strikes. It’s only this in my view that can plausibly free us from our reckless leadership (which goes beyond the current administration) and the bleak future they are preparing for us.
1. Fusion 2. Pharmaceuticals (think Ozempic economic benefit X 100) 3. Advanced weaponry (if warlike, don't build just conquer) 4. Advanced propaganda to destabilize other nations (again, if warlike)
This is mostly off the cuff, given the success of AlphaEvolve and AlphaFold I don't think they're unreasonable.
I think holding on to these beliefs about AI must give some people a sense of hope. But a hope based on reality and is stronger than one based in denial. The antidote I believe is solidarity founded on our shared vulnerability before this bleak future.
However your conclusions are what throw me off.. you kind of have this Doom and Gloom mindset which may be fair but I don't really think it's related to this particular bubble.. in other words our decline is happening off to the side of this particular bubble rather than it being caused by that particular bubble too. To me the core take away your post is that this bubble is a little bit like the Apollo program was... in a massive investment capturing a lot of people's imagination... likely lots of great things come out of it in a sense but also not clear that it all adds up in the end for a business perspective. But that's potentially okay
1. The debt bomb. Not dealing with this could cause a great depression by itself. Having it go off at the same time as we're underwater on bad capex with a misaligned economy could produce something the likes of which we've never seen.
2. We have an authoritarian president that has effectively captured the entire government, and the window to prevent consolidation of power is closing rapidly. Worse, he's an ego driven, reckless, ham-fisted decision maker with a cabinet of spineless yes men.
This is suggesting an "end of history" situation. After Fukuyama, we know there is no such thing.
I'm not sure if there is a single strong thesis (as this one tries to be) on how this will end economically and geopolitically. This is hard to predict, much less to bet on.
I mean if you are talking about USA itself falling into dystopic metastability in such a situation, maybe, but even so I think it misses some nuance. I don't see every other country following USA into oblivion, and also I don't see the USA bending the knee to techno-kings and in the process giving up real influence for some bet on total influence.
The only mechanism I see for reaching complete stability (or at least metastability) in that situation is idiocracy / idiotic authoritarianism, i.e. Trump/his minions actually grabbing power for decades and/or complete corruption of USA institutions.
My maximum is that the courts restrain Trump enough that we still have at least the shell of a democracy after he's gone. But he blazed the trail, and in 20 or 30 years someone will walk it better, having done a better job of neutralizing or politicizing or perhaps deligitimizing the courts first. Then it's game over.
You can't seriously believe that spending all your income each month while living in the country with the highest standard of living in history is "serfdom."
Hyperbolic nonsense like this makes the rest of the article hard to take seriously, not that I agree with most of it anyway.
Luxembourg, Netherlands, Denmark, Oman, Switzerland, Finland, Norway, Iceland, Austria, Germany, Australia, New Zealand, Sweden, United States, Estonia.
Based on these metrics: Quality of Life Index, Purchasing Power Index, Safety Index, Health Care Index, Cost of Living Index, Property Price to Income Ratio, Traffic Commute Time Index, Pollution Index, and Climate Index.
Source: https://www.numbeo.com/quality-of-life/rankings_by_country.j...
Sorry if I don't take numbeo.com all that seriously. If you do, you should poll people to see if they'd rather live in Oman or the US and think again.
No doubt in my mind that if you are wealthy in the US you can have a very good life, but on a whole and for most people there it’s probably fitting that it’s lower down the list for quality of life.
People are suffering, agree with the rest of what I say or not, but I can't let you slide on that.
And to this specific comment, wages outpaced inflation since the 1970s for everyone but the poorest households (I believe the bottom 10% are the exception, who I would probably agree are suffering in some sense). Working class real wage growth actually outpaced white collar real wage growth for a couple years post-COVID, for the first time in a long time. Also, wage measurements don't normally measure total compensation, notably health insurance which has been increasing much faster than wages or overall inflation for decades.
Also, there's no reason to expect wage growth to match productivity growth. Productivity gains are largely due to company investment, not increased effort from workers, and household expenses are not positively correlated with productivity metrics.
There is an additional ~30% that is notionally living paycheck to paycheck as a lifestyle choice rather than an economic necessity.
The median US household has a substantial income surplus after all ordinary expenses. There may be people suffering economically but it is a small minority by any reasonable definition of the term.
Those people have never felt real pain. They may say they have but they haven't. If this article is real and the whole shitbox falls apart, I don't think even loyalty to Trump will stop these people from finally reckoning with reality.
The GFC really set those people straight. Enough of them got desperate enough that they switched sides and voted for Obama.
No one should have been surprised. Trump has a long history of being a product person (Trump Steaks, Trump University, Trump Vodka, Trump Fragrance, Trump Mattresses, Trump: The Game etc.)
You also have to consider that the pre Web 2.0 world of the internet during the Bush years also had less opportunity of flooding the market with branded crap. You had to call a phone number or mail out a money order to order merch.
These southerners could have lost their culture, the local industry has long since left, and fentanyl is the name of the game. Yet thats still nothing compared to what Chinese (or even Russian people post USSR) had to go through.
It goes back to my point. These people in those areas think they are in for bad times but they have not seen REAL pain yet and yet they are playing games right now because they think they have nothing to lose.
Imagine if your theory comes true and the government has to cut services to the bone. No more medicare/medicaid/maintaining their infrastructure(however shit it is now it gets worse)/state subsidies for states like Nebraska.
There are probably horrors that im not even thinking of.
Good skills but wank.
Compares China's current 2025 energy capability with US at the end of 2023. Then starts throwing 2025 uptake numbers for solar and wind into the 2023 numbers as if having a TWO YEAR GAP in the numbers dont matter at all.
I read an article today in which western business leaders went to China and were wowed by "dark factories" where everything is 100% automated. Lots of photos of factories full of humanoid robots too. Mentioned only further down the article: that happens because the Chinese government has started massively distorting the economy in favor of automation projects. It's widely known that one of the hardest parts of planning a factory is figuring out what to automate and what to use human labour for. Over-automating can be expensive as you lose agility and especially if you have access to cheap labour the costs and opportunity costs of automation can end up not worth it. It's a tricky balance that requires a lot of expertise and experience. But obviously if the government just flat out reimburses you 1/5th of your spending on industrial robots, suddenly it can make sense to automate stuff that maybe in reality should not have been automated.
BTW I'm not sure the Kuppy figures are correct. There's a lot of hidden assumptions about lifespan of the equipment and how valuable inferencing on smaller/older models will be over time that are difficult to know today.
https://www.reddit.com/r/Amtrak/comments/1hnvl3d/chinese_hsr...
https://merics.org/en/report/beyond-overcapacity-chinese-sty...
It's easy to think of Uber/AirBnB style apps as trivialities, but this is the mistake communist countries always make. They struggle to properly invest in consumer goods because only heavy industry is legible to the planners. China has had too low domestic spending for a long time. USSR had the same issue, way too many steel mills and nowhere near enough quality of life stuff for ordinary people. It killed them in the end; Yeltsin's loyalty to communist ideology famously collapsed when he mounted a surprise visit to an American supermarket on a diplomatic mission to NASA. The wealth and variety of goods on sale crushed him and he was in tears on the flight home. A few years later he would end up president of Russia leading it out of communist times.
… or it is an early manoeuvre – a pre-emptive measure – to address the looming burden of an ageing population and a dearth of young labour that – according to several demographic models – China will confront by roughly 2050 and thereafter[0]. The problem is compounded by the enforced One-Child Policy in the past, the ascendance of a mainland middle class, and the escalating costs of child-rearing; whilst, culturally and historically, sons are favoured amongst the populace, producing a gender imbalance skewed towards males – many of whom will, in consequence, be unable to marry or to propagate their line.
According to the United Nations’ baseline projection – as cited in the AMRO report[1] – China’s population in 2050 is forecast at approximately 1.26 billion, with about 30 per cent aged 65 and over, whilst roughly 40 per cent will be aged 60 and over. This constitutes the more optimistic projection.
The Lancet scenario[2] is more gloomy and projects a 1 billion population by 2050, with 3 out of 10 being of the age of 65+.
It is entirely plausible that the Chinese government is distorting the economy; alternatively, it is attempting to mitigate – or to avert – an impending crisis by way of automation and robotics. The reality may well lie somewhere between these positions.
[0] https://www.populationpyramid.net/china/2050/
[1] https://amro-asia.org/wp-content/uploads/2023/12/AN_Chinas-L...
[2] https://www.thelancet.com/article/S0140-6736%2820%2930677-2/...
The most optimistic reading of this move is that it's just more of the same old communism: robotic factories seem cool to the planners because they're big, visible and high-tech assets that are easily visitable on state trips. So they overinvest for the same reason they once overinvested in steel production.
The actually most gloomy prognosis is that they're trying to free up labour to enter the army.
For something like an invasion of Taiwan or (gulp) other territories beyond that, the only way to completely subdue the captured population is with lots of soldiers.
Regarding effective conquest, we can look at the historical lesson of Rome. Conquest is effective when you can co-opt local leaders and cultures to cause them to identify with the conquering culture. Conquest that doesn't cause integration is historically unstable.
China, indeed, possesses a longstanding tradition of curating information to satisfy the sensibilities of its ruling class — a practice traceable to the dynastic courts of antiquity. Yet, to dismiss a potential adversary solely upon the architecture of its political order is — at best — ill-advised, and at worst, a grave miscalculation. The probability of threat must be judged on capacity, not narrative. Whether such an adversary proves formidable or farcical is immaterial at present — the truth will emerge within the span of a decade or so.