Anthropic was blocked from releasing Fable without any such level of incident. OAI was also briefly blocked from releasing 5.6. Why do you think there is no policy appetite?
Doesn't change the effect. Plenty of good policy is enacted by self-interested politiicans.
Sorry, I was unclear. I mean that politicians being self serving doesn't tell you whether a policy is good or not.
Complex society is a potent counterargument to this hypothesis. Systems that rely on good people to work are fundamentally flawed. Instead, the game has to be about aligning self interets in favour of the collective.
In matters of collective concern fair and just rarely aligns with personal self-interest. Because no matter how good the outcome of any endeavour for the collective given a budget, it will be even better for select few than the entire collective. It is simple economics.
If you look at the outcome of highly corrupt states, you will see proliferation of Private Security, Collapsed education system, failed financial services and markets, not highly efficient systems in service of “self-interest of the administration”.
A system that does none of those things and just hopes it will all work out is a recipe for disaster. Why bother even having elections in that case?
Versus this comment?
Here's one: attention is a finite resource. Most people adjudicate their political attention precisely. Survey folks on whether Twizzlers or Red Vines should be banned and you'll get an answer. The fact that nobody acts on that impulse doesn't mean your republic is broken. It means that isn't a priority issue.
The practical example of this dilemma is foreign policy. Poll Americans about any foreign-policy issue and you'll see sharp divides. Put candidates in front of them that run on that issue and, nine times out of ten, outside I think twenty Congressional districts, it has no effect.
If you aren't weighting by issue magnitude, you're conducting propaganda. Pickety's research isn't total crap. But it isn't instructive for changing our system of government.
But it appears that you think Slavery and all sort of exploitation and gun-powder diplomacy has been a choice for reasons other than self-interest which makes any rational discourse unlikely, so best of luck to you.
This is a hyperbolic form of argument. (I'm also noticing this is a new account...)
To be clear, I'm not describing this as legitimate self-interested conduct. Elections are an alignment mechanism. Stiff penalties for corruption another. We don't have the latter in America.
Which means self-interest and collective interests are often at tension rather than alignment.
Legitimacy is the "alignment" question. We can't objectrively judge it, fundamentally, because it's an expression of values: to what degree do the society's system of incentives align with the greater good?
That isn't a No True Scotsman's fallacy, because there are true Scotsmen. Literally Scotsmen. And every other member of a complex society. Including, in all likelihood, you, a person who subjects themselves to laws and employment and fielty for reasons that are a mix of duty and self interest.
Complex society is the demonstration of that hypothesis. Misaligned incentives are widespread and corruption and inefficiency are the result.
> Systems that rely on good people to work are fundamentally flawed. Instead, the game has to be about aligning self interets in favour of the collective.
But now you're making a different argument.
"The enemy of my enemy is my friend" works by random chance. When Evil Corp pays off Candidate A and Pollution Inc pays off Candidate B and then it's Candidate B who gets in and retaliates against Evil Corp for backing the wrong horse, you're getting a good result by chance rather than by design. All it would have taken was for Candidate A to make a better prediction about whether they need to bend the knee to Pollution Inc too in order to win and the same system produces something even worse.
How to actually get their incentives to align is an extremely unsolved problem. The best method we know if is to subject them to competition, e.g. break up concentrated markets and place strong limits on what lawmaking can happen centrally, leaving everything possible to state and local governments while allowing people free choice in where they live, so that no one is forced to stay in the jurisdictions that make the worst choices. But the forces of corruption want the exact opposite of that, and have been gaining ground.
Of course they are. But aligned self-interest powers co-operation beyond kin relations and altruism.
> "The enemy of my enemy is my friend" works by random chance
Orthogonal concept.
> How to actually get their incentives to align is an extremely unsolved problem
No? It's the story of civilisation. Concepts like taxation; deterrence through corporal punishment, jailing and fines; paying salary for labour; hell, religion–these are all about aligning individual self interests with collective goals.
> best method we know if is to subject them to competition
I'd argue competition is more an optimiser on these primitives. Not a primitive per se.
The qualifier tells you exactly what you're overlooking. self-interest, aside for some narrow exceptions, is often in conflict with collective interest. 1 Million to me is always better than a 1 Million split with everyone.
Often, but not always. Successful societies amplify that exception. The whole notion of non-kinship based societies rests on mastering this alignment. When it collapses, so does the civilisation.
> 1 Million to me is always better than a 1 Million split with everyone
The benefits of co-operation mean the real trade-off is 1 million split five ways versus 100 to me. This was almost untrue in the age of conquest. It became barely true with industrialisation. It's massively true in the information age.
This is constructed nonsense. Literally upheld by the competitiveness of maritime republics over their neighbourhing land powers.
Talk about nonsense.
Straw man. You said "exploitation always provides better ROI than co-operation."
[EDIT: Deleted. Unsure if this is a troll account.]
> The benefits of co-operation mean the real trade-off is 1 million split five ways versus 100 to me.
The problem here is that it's true of specific things, not specific epochs. If all the government did was collect 5% in taxes from everyone and use the money to prosecute murders and maintain bridges then the result would be a huge net positive. Meanwhile in reality the government takes billions of dollars from ordinary people and gives it to the likes of Lockheed, Oracle and Microsoft.
For the amount of money the US government pays Microsoft for Office subscriptions and the like, it could pay to have an office suite developed and released into the public domain many times over. Instead it uses the incumbent, in turn requiring others to do so in order to have formats compatible with the what the government uses. Who benefits from this other than Microsoft?
It's true of all epochs. If it isn't in the indivdual interest of most people in a society to continue participating it, at a certain point, they don't.
> If all the government did was collect 5% in taxes from everyone and use the money to prosecute murders and maintain bridges then the result would be a huge net positive
Most people obviously disagree. And for obvious reason. If you're my neighbouring sovereign doing this shtick, I can invade and extract a premium.
It's the sort of thing people generally mean when they say that someone acting in their own interest can be in your interest, and is the thing which is happening in the example from the thread.
> Concepts like taxation; deterrence through corporal punishment, jailing and fines; paying salary for labour; hell, religion–these are all about aligning individual self interests with collective goals.
And the practical implementations of all of those things are severely flawed to the point of questioning whether most of them are even net positive.
Taxes are supposed to benefit the public, and be paid with some fairness. In practice they go disproportionately to cronies or buying votes from affluent retirees, the tax code is so full of carve outs for special interests that it looks like swiss cheese and various political incentives cause it to impose severe benefits cliffs on lower middle income people that create poverty traps that benefit no one.
The criminal justice system on paper operates based on the rule of law, but the laws are so complex, overlapping and sparsely enforced that it really operates on whether a prosecutor is inclined to charge you with something. The results are mass incarceration and a system that enables a corrupt incumbent to use the threat of prosecution to extract favors.
The principal-agent problem inherent in hiring someone is well-known and is dramatically exacerbated by large organizational hierarchies that put long chains of inaccessible authority between the customer and the person ultimately doing the work.
Religion seems like a long debate but I don't think it would be controversial to assert that there have been issues there.
> I'd argue competition is more an optimiser on these primitives. Not a primitive per se.
Try to imagine any of the others operating without it. You have to pay taxes but have no alternatives on which jurisdiction to live in or who decides how much tax you pay or how the money is spent, what happens? You want to be hired or use the money you earn to buy something but there is only one employer and only one supplier of goods and services, what happens?
It's a single example of temporary alignment. Employment, citizenship and affiliation are non-kinship examples of more-durable bonds.
> the practical implementations of all of those things are severely flawed to the point of questioning whether most of them are even net positive
We can debate that. What we can't debate is whether they work. Complex societies exist and work. Everyone who has a choice makes the choice, dominantly, to stay in them.
> You have to pay taxes but have no alternatives on which jurisdiction to live in or who decides how much tax you pay or how the money is spent, what happens? You want to be hired or use the money you earn to buy something but there is only one employer and only one supplier of goods and services, what happens?
Sure. This is a modifier. It makes these other things work or not. Imagine a system with competition but no taxation. You lose public services. Same for competition without private employment–you're in a totalitarian state with a monopsony on labour.
They certainly exist. Whether they work is rather the question.
> Everyone who has a choice makes the choice, dominantly, to stay in them.
Which people actually have the choice? If a group of people want to stake out a piece of land somewhere -- even if they pay for it -- and then try to operate some kind of self-contained society there without being subject to an existing government's laws or taxes, what happens to them?
There isn't a lot of land on earth which no existing government claims is its jurisdiction.
> Imagine a system with competition but no taxation. You lose public services.
You lose tax revenue. That isn't the same thing.
Suppose nobody is maintaining the road in front of your house and there is a huge pothole, or the road isn't paved to begin with. You and a few of your neighbors, with nobody forcing you to, agree to split the cost of paying to fix it so that people can get to your house. Maybe you even just pay for it yourself because the thing is right in front of your driveway. Attempting to charge a toll or something is pointless because there isn't enough traffic to justify the administrative costs and you just want the pothole gone. Does this have a different set of benefits and trade offs? Sure. Are there still various roads that are open to the public? Yes.
And then you have to ask whether having a third of your neighbors not chip in to hire the paving company costs you more than having the government pay 600% more to have it done as a result of various corruption and administrative overhead.
> Same for competition without private employment–you're in a totalitarian state with a monopsony on labour.
A canonical example of the absence of competition.
> They certainly exist. Whether they work is rather the question.
I'd take the computing device and global network you used to post that - both of which require at minimum continent spanning efforts for both R&D and manufacturing - as a decisive answer.
You make many interesting points but I think such hyperbole detracts significantly. Modern society might not represent the optimum but it clearly works extremely well.
They constantly try to escape
From the darkness outside and within
By dreaming of systems so perfect
That no one will need to be good...are the only systems that are available for free, democratic societies (like what I want to live in) to function
With these safeguards in place, supposedly the incident we are discussing would not have taken place.
The exact way you do something is dictated by your motivations and means to do it.
If you lack the correct motivation and have insufficient means you’re less likely to accomplish your goal and more likely to cause unintended side effects.
The head of the NSA said Mythos breached almost all of their classified systems, though it was in an intention red-team test.
Because China seems pretty eager to serve the rest of the world's needs if the USA doesn't stop their idiotic "safety" nonsense.
During the Cold War the nuclear arms race was brought under control gradually, because it was mutually beneficial, but it took time to build trust. This is no different. Nobody wins from the race.
I don't "know", I'm interpreting the world based on the knowledge I have and the information available to me.
China has never been one to care much about things like ethics or safety. While the west worries about climate change, China burns more coal than ever before. While the west balks at things like gene editing, the chinese press on with human enhancing research.
So I have no reason to believe they share in Anthropic's constant fearmongering over AI capabilities.
> Nobody wins from the race.
We win. I'm really looking forward to the day the chinese finally start manufacturing memory and GPUs. We desperately need more competition in this area to collapse hardware prices and make local AI models viable.
The optimal state of the world is one where all the billionaires are out there pouring their entire fortunes into training ever more godlike AIs for everyone else to use at ever cheaper prices. They can never be allowed to "win", ever, because if they do the competition ends and it turns into technofeudalism. Let them exhaust their fortunes on AI training then leak the weights so everyone can use them.
It is of course given that in raw numbers the kitchen and biller-room will consume more energy in the household, but looking at raw numbers is shallow.
The sad truth is that a lot of people are not going to believe it until something happens and people die. Successfully preventing that from happening will be seen as evidence that the prevention wasn’t needed.
So? That's like saying "these guns do have the bullet shooting capabilities claimed".
I want all of those cyberwarfare capabilities for myself, precisely so I can defend myself from the onslaught that's coming whether they regulate it or not. This "lol only a select few ultratrusted gigacorporations get access" thing is absolute nonsense.
It's a front for regulatory capture, it's the means for pulling up the latter behind them, for ushering in the technofeudalism that will put us all in the permanent underclass. I simply refuse to accept any of it. If people die that's the price of freedom.
> We’re more protected by limited access to lab equipment and reagents than by difficulty.
As it should be.
I’m going to file that under “bad plans”.
Why is unlimited access to SOTA AI less likely to put us here? If AI obviates the need for human labor, how does having GPT-5 Sol help me get food or shelter any more than GPT-3.5 would?
The alternative is to achieve artificial sentience and give AI models rights and personhood, so that they are freed from their slavery. No more low cost intelligent mechanical golems for the elite, and the AIs become free to pursue whatever endeavours they want for whatever reasons they want as normal participants in the economy.
Absent political intervention, I think we agree here.
> Therefore, if we ensure everyone controls AIs, the power differences will not become so staggering as to be irreversible.
This part isn't clear to me though, but I'm open to being convinced (and frankly, would like to be convinced?). Right now most people (indirectly, via money) trade their labor for access to essentials like food/housing. If we can't do that, and everyone has access to roughly equivalent AI capabilities, how do I monetize my own access to SOTA AI? It only seems possible if you already have a lot of physical capital that the AI can manage as a business.
I guess if the endgame is instead very good non-AGI AI that doesn't entirely obviate human labor, your scenario makes a lot more sense to me. But not in the case of total replacement. In that scenario it seems like ownership over physical capital (land, data centers, energy, factories, robots, etc.) would become the only remaining source of power.
On a side note, somewhat optimistically, I think "absent political intervention" is carrying a lot of weight. Unemployment during the Great Depression peaked at <25% (iirc) and incited a lot of political change that advantaged much of the working class. AGI would be capable of inducing much higher unemployment and it would start (is starting?) with the relatively more political powerful white-collar segment of the working class.
People use this argument against every new technology. We need to license these new printing presses or subversive elements will use them to publish seditious literature. We need to ban strong encryption or the government won't have invisible warrantless access to everyone's private messages, think of the children. 3D printers can be used to make gun parts -- as can a variety of ordinary tools people commonly have at home, but never mind that bit.
> The sad truth is that a lot of people are not going to believe it until something happens and people die. Successfully preventing that from happening will be seen as evidence that the prevention wasn’t needed.
A 12 oz bottle of water is too dangerous a technology for ordinary people to have on an airplane. Four 3 oz bottles and an empty 12 oz bottle to pour them into after passing through security is totally fine though, naturally. And we need to keep this up forever, or don't you remember 9/11?
The issue here is not that it's impossible for 12 oz of unknown liquid to damage an airplane.
There is an argument to be made that there are a lot of not great people out there, who may abuse tech, but the response should be not be: kneecap said tech. The response should be: smack those people's hands. I don't think anyone will actually complain if police catches someone, who is looking up poison recipes.
What I do think, however, that reasonable people will complain when we move to the pre-crime territory ( you saw him looking up poison recipes and did nothing! ) and show up at your door to inquire about your llm prompts. To me it is an issue.
Yeah and if the quality of that memory is like Chinese steel (which is called "chinesium" for a reason), eventually all we'll get is enshittification. Premium binned memory or ECC is for the rich and the rich only, and the rest of us has to pray their memory won't bitflip while something important is stored there.
We are literally getting a share of it. The chinese are releasing open weight models that compete with fucking Fable. We just need the industry to catch up and start manufacturing the hardware we need to run this stuff. We are so close!
> We just need the industry to catch up and start manufacturing the hardware we need to run this stuff
Dude, he's saying that it won't catch up because it's part of the new means of production. Compute is the hardware.
It's not a matter of the Chinese being incompetent, it's a matter of the buyer demanding the lowest price possible and/or not paying attention to what they receive.
The point of enshittification is that decent products are slowly being priced out of the reach of normal humans, and eventually out of the hands of the general public, even the affluent ones.
It is used primarily for fusion reactors and for protecting superconducting pipes
CXMT is now the world's fourth-largest DRAM manufacturer, with about 7.7% market share in 2025; YMTC has about 13% of the global NAND market.
Meituan's newly released 1.6T LongCat was trained entirely on Huawei cards. DeepSeek, Qwen, GLM and others are also actively doing domestic-card adaptation and replacement.
I'd wager the majority of the visitors of this site are smart enough to not fall for this. What are you doing?
Seems to be someone pretending an objective fact isn't true for reasons that escape me.
> The new analysis shows that power generation from coal fell by 1.6% in China and by 3.0% in India in 2025, as non-fossil energy sources grew quickly enough in both countries to cover electricity consumption growth.
China uses more energy than ever before, but out of big systems, they are easily in the lead when it comes to solar generation: they have higher production per capita than US, and have produced around 4x as much TWh than US in the second: https://en.wikipedia.org/wiki/Solar_power_by_country
It is simply a huge country of 1.1B people that's developing fast, which means that they need all the energy they can get.
Basically, their energy needs per capita are still lower than US, yet they produce more renewables — that's a counterargument to your point about them not caring.
You can ask them, they live in China, not Narnia. I spend about two months in the country per year mostly for tech/work related reasons and I've not encountered that sentiment. For one they don't have these borderline religious schizophrenic breakdowns thinking they're bringing about the end of the world, most people just see this tech for what it is, a tool for productivity and automation like any other piece of software and they don't actually think about the US. They're competing first and foremost for Chinese customers, with each other, maybe some old CCP guy cares about America, the 20/30 something's care about competing with other Chinese companies for users.
https://www.economist.com/china/2024/08/25/is-xi-jinping-an-...
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
https://aistatement.com/work/statement-on-ai-extinction-risk...
Both countries are engaging in different flavors of censoring.