Is Tallinn sincere about holding $10bn of Anthropic, who refuse to slow down until everyone else slows down.
Then funding PauseAI, who protest outside the AI companies?
He is funding protests against the thing he owns.
Then funding PauseAI, who protest outside the AI companies?
He is funding protests against the thing he owns.
If he thinks the AI is dangerous or even omnicidal, that makes him an omnicide profiteer. That makes PauseAI funded by omnicide profiteering.
If he has no "controlling interest that would allow him to force it to pause", he can't even excuse the investment on the basis that it is helping to make the company more safe.
He is just profiting from the Anthropic's active development and sale of AI; if you think the companies should pause, it is clearly hypocritical to be living large at the expense of others.
Is there anything at all about this argument that is false, I think it is just straightforwardly true.
Asking for regulation is suspicious. Asking for no regulation is suspicious. At some point you have to stop worrying about these guys motives and just do what is best for society
Note the Anthropic scaling policy. I am taking care not to take quotes out of context. This is an accurate excerpt.
"This section outlines our recommendations for what it would take, at an industry-wide level, to keep catastrophic risks reliably low through a period of rapid advances in AI capabilities." [...]
"The right column describes our recommendations for industry-wide safety at each threshold." [...]
"In particular, we cannot unilaterally and unconditionally commit to staying in line with the industry-wide recommendations in the right column." (p4) [https://www-cdn.anthropic.com/e670587677525f28df69b59e5fb4c2...]
They refuse to act safely if it would cause them to fall behind in the industry.
"We hoped that by the time we reached these higher capabilities, the world would clearly see the dangers, and that we’d be able to coordinate with governments worldwide in implementing safeguards that are difficult for one company to achieve alone." [https://www.anthropic.com/news/responsible-scaling-policy-v3]
They will not act safely unless they are able to collude with other firms to set production quotas.
This is a formal declaration that Anthropic will not slow down according to what they consider to be safe unless they are able to form a cartel.
A cartel is illegal.
To create the cartel, Anthropic must pursuade the government to make coordinated production legal. To make the case for the cartel, Anthropic relies on safety. They are blackmailing the entirety of the world by threatening to proceed at an unsafe pace, unless they are granted their cartel.
Let's play a game. Prove that you are not a power seeking AI looking to stop regulation in order to ensure the race continues. See, two can play this game of throwing random claims around.
>They will not act safely unless they are able to collude with other firms to set production quotas.
And? Neither will OpenAI, nor will any of the major players. Hell, there isn't even much legal precedent on what "safely" even is here. This is not a cartel, it's asking the government to make a set of laws and rules for everyone to play under otherwise the entire system ends up being a race to danger.
The people in Anthropic were thinking about AI safety when you were still in diapers. Not everything is a vast conspiracy.
>"random claims"
You don't explain how they are random, I think they are true.
> [They will not act safely unless they are able to collude with other firms to set production quotas.] "And? Neither will OpenAI, nor will any of the major players."
They should act safely. I think it is important to act safely at all times. That is based on the notion of being responsible for one's own actions, regardless of the actions of others. It does not follow from the fact that there is a bandwagon, that joining the bandwagon is the right thing to do.
Example: "It was right to steal from the shop, because everyone else was stealing."
This is clearly non sequitur.
Something else must be imported and in Anthropic's case it is just consequentialism, which states that if they develop AI in an unsafe way and take the lead, it will be a lesser evil than another firm taking the lead.
Unfortunately, consequentialism is illegal in the USA. It is not permissible to risk the lives of others because you think it will be best for everyone. That stems from a reading of human rights to life that can't be willed away by appealing to the consequences, it is the law that Anthropic does not have the authority to impose these risks on others.
The US Government sometimes acts in a consequentialist way, for example shooting down hijacked airplanes. That is mostly because it has authority grounded (in theory) in a mandate from the people such that it makes exceptions to the law. Anthropic has ZERO authority to make these exceptions.
"Hell, there isn't even much legal precedent on what "safely" even is here."
They can develop it safely. If they can't work it out, they can just stop developing it completely. The only cost to them is personal profit and gain.
"This is not a cartel, it's asking the government to make a set of laws and rules for everyone to play under otherwise the entire system ends up being a race to danger.
By definition, coordinated slowing down is a cartel.
"The people in Anthropic were thinking about AI safety when you were still in diapers."
Again non sequitur. If Anthropic has been thinking about this for longer this simply does not show that they are more correct.
"Not everything is a vast conspiracy."
This is also non sequitur. If not everything is a conspiracy, this does not show that what I am saying is wrong.