There's nothing special about frontier LLM companies. Singling them out for special financial restrictions is a stupid idea based on nothing but your own irrational and uninformed prejudices. No actual harm has been demonstrated. No one has died.
On the one hand, yes, they're companies like any other.
On the other hand, I can count on one hand the number of companies that have publicly declared «We're working on WMDs [0], we don't think we are capable of working on them safely, and we refuse to stop working on them. However, if we get special legal and regulatory treatment we'll be quite happy to put in the stop work order.».
So, yeah, there are some special things about the major LLM manufacturers and none of them are good.
[0] Anything that has a 10% chance of suddenly destroying all of humanity is a WMD.
The source for this claim is the lead of Anthropic's Alignment Science division. His actual claim is that the risk is greater than 10%. From [0]:
Jacob['s claim that "the people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt."] is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. ...
Given that the major LLM manufacturers are actively using these claims of the imminent and total destruction of humanity to extort concessions from lawmakers and regulators, we are making a massive error if we do not take their claims to their logical conclusion... namely that the major LLM manufacturers are deliberately and knowingly threatening the extermination of all of humanity in order to enrich their management, employees, and investors. If you or I were in a position to credibly threaten that level of destruction, we and all of our associates would rapidly get disappeared to a CIA black site and tortured until the end of our deliberately-medically-extended lives.
To be crystal clear, I'm certain that their claims of likely, imminent doom are utter bullshit. I'm very confident that they're claiming to be holding all of humanity at nukepoint in an attempt to get Uncle Sam to give them an escape from investors who are soon to be massively angry because
* they've hit a massive increase in the cost required to get better models
* a huge fraction of the GPUs that have been purchased to power this project are sitting in warehouses, and it's only a matter of time before the right people start asking why all that money was so ill-spent
* they have been and continue to lie so much about how much datacenter capacity is coming online in the near future, and that those claims are lies is becoming obvious
* they habitually lie about the capabilities of their newly-released products
* they're staring down the barrel of so many lawsuits that aim to hold them to account for the shitty and illegal behavior they've engaged in over the past many years
But the US Congress not only is almost entirely uninformed about how this shit actually works, the FISA Amendments Act of 2008 fiasco demonstrated that -if you get them scared enough- Congress has no problem making behavior that's plainly illegal and obviously a huge bucket of civil liberties violations retroactively legal... instantly mooting all ongoing lawsuits regarding the illegal and unconstitutional conduct. That is the big prize that the major LLM manufacturers are after.
I have articulated my logic up and down this thread with specific premises and detailed explanations on why the conclusions follow from them. You may not agree but you don’t have justification to say they are irrational.
I am singling out the frontier labs because they have created a singularly unique technology that presents potential and actual damage that is…singular. They have disclosed hacking through coordinated autonomous agents that would have sent human hackers to jail but will not result in any similar result because the law isn’t presently able to apply to agents or the labs.
My arguments are not prejudice, I have thought deeply about this as I have personally developed multiple AI deployments in a variety of settings. I studied philosophy, cognitive science in undergrad, and grad school and have a law degree. I have been a CTO, CPO and CEO for startups and national health companies. I personal build AI agents that handle thousands of daily conversations with health care workers and patients, I built an agentic IDE for our dev team with a custom harness capable of self defining tools and calling them in a custom cloud Linux environment, I personally build our internal hardware inference stack and orchestration API. And I have personally sponsored academic research through my foundation with researchers are Duke madlab, northwestern, UCF, CM and case western on a range of topics such as perception of empathy and its effects on humans in healthcare, guardrail design for ethical deployment and alignment, moral reasoning ability, etc. I am not a doomer or an accelationist. I am responding specifically to current evidence of how the models presently work and how the corporate governance of the Labs is effectuating their power and vision.
Or maybe you could reason from first principles, look at the premises I have outlined and draw causal inferences to extrapolate the possible outcomes.
Premise 1: AI can plan a series of actions that include broad computer and internet use, 2: AI can execute them autonomously without human oversight, approval or ability to undo the effects, 3: no tool or software has ever been able to autonomously plan, execute and iterate in this way. Therefore AI is a materially different technology than has existed.
Premise 4: knowledge work involves analyzing information, devising plans in response to goals from that information, and executing those plans, reporting and coordinating with other knowledge workers, 5: AI ability to execute knowledge work has demonstrably increased over the last 24 months, 6: all knowledge work can plausibly be described and defined in ways AI can do given current abilities. 7: electricity and the ICE reduced human labor for physical movement, 8: AI will similarly reduce human labor for knowledge work. 9: knowledge work is of similar scale and scope to the tasks that were automated during the Industrial Revolution. Therefore it’s plausible that AI will transform society to an extent similar to the Industrial Revolution.
Premise 10: Some percentage of the actions designed and executed by AI could be harmful to databases, or the stable operation of software systems, 11: some percentage of those systems are critical to the stability of society. Therefore some percentage of agentic actions presently possible may damage society.
12. There exists a threshold of instability from which society will not recover such as supply chain disruption, mass unemployment, perceived military action (false or real) that triggers preemptive strikes, 13: certain actions AI is presently capable of doing could cross the instability threshold, 14: AI could knowingly or unknowingly trigger these types of actions, 15: some social disruption could result in full scale collapse or human extinction, 16: only biological weapons and nuclear weapons approach this level of destruction, therefore AI presents at least the level of risk as those technologies.
It’s fascinating that after multiple comments predicting post-capitalist utopias and the complete collapse of the university system, your line in the sand for rigor is demanding academic peer-review to prove that agentic AI presents systemic risk.