Dario Amodei calls OpenAI’s messaging around military deal ‘straight up lies’

This is my first thought as well. It's too obvious. He should have consulted ChatGPT before the announcement.

I believe this understanding is correct. The issue many people have these days with Dept. of War, and most of Trump admin is that they have little respect for laws. They only follow the ones they like and openly ignore the ones that are inconvenient.

Dept of "War" should have zero problems agreeing to the two conditions Anthropic outlined, if they were honest brokers. But I think most of us know that they are not. Calling them dishonest brokers seems very charitable.

He also claimed that they would build rules into the model the DoD would use, preventing misuse. Aka he claims OpenAI will quickly solve alignment and build it right in...I wouldn't hold my breath.

[1] https://privacy.openai.com/policies?modal=take-control

Just good 'ol fashion grifting mixed with a bit of government corruption.

This country has been boiling the frog of graft, grifting, and corruption too long.

Secret FISA court decisions will say the use is lawful, but you’ll never get to read or challenge those decisions.

I think it’s also possible DoW didn’t care about the conditions but just wanted some pretext to punish Anthropic because Dario isn’t a Trump boot licker like the rest of the SV CEOs.

But besides Sam Altman, this whole episode has made me totally and completely lose all respect for Paul Graham. I used to really idolize pg, and I really used to like his essays, but over the years I've found his essays increasingly displayed a disturbing lack of introspection, like they'd always seem to say that starting a startup is the best thing anyone can do, and if you're not good at startups then you kind of suck.

But his continued support of Altman in this instance (see https://x.com/paulg/status/2027908286146875591, and the comment in that thread where he replies "yes") is just so extra disappointing and baffling. First, his big commendation for Altman is that he's doing an AMA? Give me an f'ing break. When someone is a great spin doctor I'm not going to commend them for doing more spinning. It's like he has total blinders on and is unwilling to see how sama's actions in this instance are so disgusting and duplicitous. Maybe subconsciously he knows he's responsible for really launching sama into the public consciousness, so he now just is incapable of seeing the undeniably shitty things sama has done.

Oh well, I guess it's just another tech leader from the late 90s/early 00s who has just shown me he's kind of a shitty person like a lot of us.

And while this administration is brazen about this, it's not really a drastic change anywhere.

In fact most EU laws (GPDR, AI regulation, Chat Control) are directly, up front, declaring they themselves won't respect it. They very directly have one set of rules for states, government employees, ... and ANOTHER set of rules for everyone else. And they're incredibly brazen. For private individuals, companies it goes very far, it's essentially impossible to even know what does and does not violate the GPDR, and you can't ask the courts, that's not allowed. You also cannot use the courts to compel government to do anything under these laws.

For governments, when it comes to what's allowed, it goes incredibly far. Governments can declare any action legal under the GPDR, before and after the fact, without parliament involvement. It does not matter if that action was done by the government themselves, or if it's an action by a private company (so the government can use subcontractors for any violation of the GPDR)

This means that, for THE example given for GPDR protection: medical information. Medical insurance in the EU is either state-owned or has exceptions, the law does the exact opposite of what it appears to do: it makes all your medical information available for medical insurers. And the police (e.g. to find you). And the tax office. And courts. And medical institutions themselves (to deny transplants to smokers). And ... And while doctors (and priests) used to be huge no-no's when it came to information gathering, that's no longer the case. If a doctor uses the state required medical file, your medical information flows straight into a state database, immediately searchable for everyone the GPDR supposedly protects you against.

Verification requires access to classified logs. These logs would attract the spies of the whole world. Even if these logs are in principle for "past actions", in practice past logs (for war games, for example) would compromise future strategy.

Since these manual audits are too risky, the only alternative is to hard-code limits into the AI. But are we ready trust an AI to "judge" a mission and refuse to execute during a crisis?

Anthropic wanted technical enforcement, the Pentagon wanted trust.

It’s a choice between two bad options: an unaccountable military and an unreliable AI kill switch. They are both very dangerous, just in different ways.

> However, only an act of Congress can legally and formally change the department's name and secretary's title, so "Department of Defense" and "secretary of defense" remain legally official.

https://en.wikipedia.org/wiki/United_States_Department_of_De...

> “[We will] tailor use restrictions to the mission and legal authorities of a government entity” based on factors such as “the extent of the agency’s willingness to engage in ongoing dialogue,” Anthropic says in its terms. The terms, it notes, do not apply to AI systems it considers to “substantially increase the risk of catastrophic misuse,” show “low-level autonomous capabilities,” or that can be used for disinformation campaigns, the design or deployment of weapons, censorship, domestic surveillance, and malicious cyber operations.

Source: https://techcrunch.com/2024/11/07/anthropic-teams-up-with-pa...

Just to nitpick, Palantir isn't doing surveillance like Flock. They do data integration the way IBM does under contract for the governments. Some data pipelines include law enforcement surveillance data which get integrated with other software/databases to help police analyze it. There's no evidence they are collecting it themselves despite recent headlines. It's a relatively minor but important distinction IMO.

https://www.wired.com/story/palantir-what-the-company-does/

OpenAI claims their terms of service for DoD contain the same limitations as Anthropics proposed service agreement. Anthropic claims that this is untrue.

Now given that (a) the DoD terminated their deal with Anthropic, (b) stated that they terminated because Anthropic refused modify their terms of service, and (c) then signed a deal with openAI; I am inclined to believe that there is in fact a substantial difference between the terms of service offered by Anthropic and OpenAI.

“I think this attempted spin/gaslighting is not working very well on the general public or the media, where people mostly see OpenAI’s deal with DoW as sketchy or suspicious, and see us as the heroes.... It is working on some Twitter morons, which doesn’t matter, but my main worry is how to make sure it doesn’t work on OpenAI employees. Due to selection effects, they’re sort of a gullible bunch, but it seems important to push back on these narratives which Sam is peddling to his employees.”

Edit: Also openly calling OpenAI employees "gullible" and "twitter morons" seems sub-optimal if you like that talent to work for you at some point.

Example - https://x.com/tszzl/status/2029334980481212820

> Graham was immediately impressed by Altman, later recalling that meeting the 19-year-old felt like what it must have been like to talk to Bill Gates at the same age. He noted Altman's intense "force of will" from their early interactions.

Is there a Gates-like "presence" or a "force of will" displayed in his public interviews?

I read that quote and see no positive interpretation. It was always a negative description.

I think maybe this community could use a bit more natural skepticism of hierarchy.

Anthropic actually partnered up with Palantir. They are not the saints you think they are, either.

We should stop worshipping people and companies and stop putting them on pedestals. Just because one party is at fault, doesn't mean the other is automatically innocent. These are all for-profit companies at play here.

https://investors.palantir.com/news-details/2024/Anthropic-a...

Which of these two CEOs wants to have an unelected spot in the decision loop of our government?

Once I dug into this story, I realized that only one of these companies was attempting a real power grab. Maybe the EAs are doomed to try coup after coup and lose every time.

The SCR part is excessive, though, especially if it's interpreted broadly. Altman gets credit for sticking up for Anthropic on that point, but not much credit, because it's so obvious that it's overkill.

The guy can lie with a perfectly straight face. He's the kind of person who tells another lie just to cover the last one, and then another to cover that.

Meanwhile he keeps making everyone more and more dependent on him, so by the time people finally realize what's going on, they can't afford to push him out.

Also if you've ever actually chatted with anyone at the company you'd know that they are not all the same and Anthropic genuinely does stand apart here.

Could you point me to one other $300B+ company that would be willing to do this?

I do agree with your point that Amodei is playing a game though. Whether he’s winning the bigger picture or not it’s unclear. His red lines are already so watered out, like how domestic surveillance is not ok, but international? totally fine.

I don't know how reliable that source is. In any case, here's the text from that link, for posterity:

"I want to be very clear on the messaging that is coming from OpenAI, and the mendacious nature of it. This is an example of who they really are, and I want to make sure everything sees it for what it is. Although there is a lot we don’t know about the contract they signed with DoW (and that maybe they don’t even know as well — it could be highly unclear), we do know the following:

Sam’s description and the DoW description give the strong impression (although we would have to see the actual contract to be certain) that how their contract works is that the model is made available without any legal restrictions ("all lawful usee") but that there is a "safety layer", which I think amounts to model refusals, that prevents the model from completing certain tasks or engaging in certain applications.

"Safety layer" could also mean something that partners such as Palantir tried to offer us during these negotiations,which is that they on their end offered us some kind of classifier or machine learning system, or software layer, that claims to allow some applications and not others. There is also some suggestion of OpenAI employees ("FDE’s") looking over the usage of the model to prevent bad applications.

Our general sense is that these kinds of approaches, while they don’t have zero efficacy, are, in the context of military applications, maybe 20% real and 80% safety theater. The basic issue is that whether a model is conducting applications like mass surveillance or fully autonomous weapons depends substantially on wider context: a model doesn’t "know" if there’s a human in the loop in the broad situation it is in (for autonomous weapons), and doesn’t know the provenance of the data is it analyzing (so doesn’t know if this is US domestic data vs foreign, doesn’t know if it’s enterprise data given by customers with consent or data bought in sketchier ways, etc).

The kind of "safety layer" stuff that Palantir offered us (and presumably offered OpenAI) is even worse:our sense was that it was almost entirely safety theater, and that Palantir assumed that our problem was "you have some unhappy employees, you need to offer them something that placates them or makes what is happening invisible to them, and that’s the service we provide".

Finally, the idea of having Anthropic/OpenAI employees monitor the deployments is something that came up in discussion within Anthropic a few months ago when we were expanding our classified AUP of our own accord. We were very clear that this is possible only in a small fraction of cases, that we will do it as much as we can, but that it’s not a safeguard people should rely on and isn’t easy to do in the classified world. We do, by the way, try to do this as much as possible, there’s no difference between our approach and OpenAI’s approach here.

So overall what I’m saying here is that the approaches OAI is taking mostly do not work: the main reason OAI accepted them and we did not is that they cared about placating employees, and we actually cared about preventing abuses. They don’t have zero efficacy, and we’re doing many of them as well, but they are nowhere near sufficient for purpose. It is simultaneously the case that the DoW did not treat OpenAI and us the same here.

We actually attempted to include some of the same safeguards as OAI in our contract, in addition to the AUP which we considered the more important thing, and DoW rejected them with us. We have evidence of this in the email chain of the contract negotiations (I’m writing this with a lot to do, but I might get someone to follow up with the actual language). Thus, it is false that "OpenAIs terms were offered to us and we rejected them", at the same time that it is also false that OpenAI’s terms meaningfully protect them against domestic mass surveillance and fully autonomous weapons.

Finally, there is some suggestion in Sam/OpenAI’s language that the red lines we are talking about, fully autonomous weapons and domestic mass surveillance, are already illegal and so an AUP about these is unnecessary. This mirrors and seems coordinated with DoW’s messaging. It is however completely false. As we explained in our statement yesterday, the DoW does have domestic surveillance authorities, that are not of great concern in a pre--AI world but take on a different meaning in a post-AI world.

For example, it is legal for DoW to buy a bunch of private data on US citizens from vendors who have obtained that data in some legal way (often involving hidden consents to sell to third parties) and then analyze it at scale with AI to build profiles of citizens, their loyalties, movement patterns in physical space (the data they can get includes GPS data, etc), and much more.

Notably, near the end of the negotiation the DoW offered to accept our current terms if we deleted a specific phrase about "analysis of bulk acquired data", which was the single line in the contract that exactly matched this scenario we were most worried about. We found that very suspicious. On autonomous weapons, the DoW claims that "human in the loop is the law", but they are incorrect. It is currently Pentagon policy (set during the Biden admin) that a human has to be in the loop of firing a weapon. But that policy can be changed unilaterally by Pete Hegseth, which is exactly what we are worried about. So it is not, for all intents and purposes, a real constraint.

A lot of OpenAI and DoW messaging just straight up lies about these issues or tries to confuse them.

I think these facts suggest a pattern of behavior that Ive seen often from Sam Altman, and that I want to make sure people are equipped to recognize:

He started out this morning by saying he shares Anthropic’s redlines, in order to appear to support us, get some of the credit, and not be attacked when they take over the contract. He also presented himself as someone who wants to "set the same contract for everyone in the industry" — e.g. he’s presenting himself as a peacemaker and dealmaker.

Behind the scenes, he’s working with the DoW to sign a contract with them, to replace us the instant we are designated a supply chain risk. But he has to do this in a way that doesn’t make it seem like he gave up on the red lines and sold out when we wouldn’t. He is able to superficially appear to do this, because (1.) he can sign up for all the safety theater that Anthropic rejected, and that the DoW and partners are willing to collude in presenting as compelling to his employees, and (2.) the DoW is also willing to accept some terms from him that they were not willing to accept from us. Both of these things make it possible for OAI to get a deal when we could not.

The real reasons DoW and the Trump admin do not like us is that we haven’t donated to Trump (while OpenAI/Greg have donated a lot), we haven’t given dictator-style praise to Trump (while Sam has), we have supported AI regulation which is against their agenda, we’ve told the truth about a number of AI policy issues (like job displacement), and we’ve actually held our red lines with integrity rather than colluding with them to produce "safety theater" for the benefit of employees (which, I absolutely swear to you, is what literally everyone at DoW, Palantir, our political consultants, etc, assumed was the problem we were trying to solve).

Sam is now (with the help of DoW) trying to spin this as we were unreasonable, we didn’t engage in a good way, we were less flexible, etc. I want people to recognize this as the gaslighting it is.

Vague justifications like "person X was hard to work with" are often used to hide real reasons that look really bad, like the reasons I gave above about political donations, political loyalty, and safety theater. It’s important that everyone understand this and push back on this narrative at least in private, when talking to OpenAI employees.

Thus, Sam is trying to undermine our position while appearing to support it. I want people to be really clear on this: he is trying to make it more possible for the admin to punish us by undercutting our public support. Finally, I suspect he is even egging them on, though I have no direct evidence for this last thing.

I think this attempted spin/gaslighting is not working very well on the general public or the media, where people mostly see OpenAI’s deal with DoW as sketchy or suspicious, and see us as the heroes (we’re #2 in the App Store now!). Itis working on some Twitter morons, which doesn’t matter, but my main worry is how to make sure it doesn’t work on OpenAI employees.

Due to selection effects, they’re sort of a gullible bunch, but it seems important to push back on these narratives which Sam is peddling to his employees."

For that matter, explain why the Pentagon would balk at not spying on every American.

Where autonomous transformer-based munitions will be used are basically "here is a photo of a face, find and kill this human" and loitering munitions will take their time analyzing video and then decide to identify and attack a target on their own.

EDIT: Or worse: "identify suspicious humans and kill them"

In a world where LLMs produce very convincing but subtly wrong output, this makes me uncomfortable. I get that warfare without AI is in the past now, but war and rules of engagement and AI output etc etc etc all seem fuzzy enough that this is not yet a good call even if you agree with the end goals.

Its not fully autonomous ice cream machines, its fully autonomous _weapons_. are you stupid or are you dumb? I don't think you're asking an honest question.

Also, that very much sounds like the government knows best and citizens should just trust it unconditionally.

The government asks if they can rent your car. I hope we agree that you don’t have to say yes. (Specific exceptions exist to places of lodging etc.)

Anthropic is exercising their right to say no in the same way.

Misunderstanding of what is happening. They have terms and conditions with their private property that anyone can choose to accept or decline. The DoD wants to them turn around and say these terms for a private company's contract around licensing of their private property are so egregious that the government and all government contract holders should be forced out of using any products by that company

(FWIW I am with you; I haven’t found a local model that works well enough to be a daily driver)

You mean other than Nuremberg which was focused on prosecuting war criminals? Remember, IBM was never held to account for propping up the Nazi war and genocide machine. People who made Zyklon B weren't either. Corporate governance is the instrumental convergence of our time.

I’m sure anthropic has signed up more revenue this week in response to this debacle to cover it. Where they’re actually screwed is if the gov follows through and declare anthropic a supply chain risk.

As opposed to all those famous ethical battles where there's nothing in it for you to do the wrong thing?

Not a chance. The DoD has massive pockers which and INCREDIBLY SPREAD OUT. You can't underestimate how spread this money is. The DoD has maybe a 64 GPU cluster and ALMOST NO ONE USES IT FOR DEEP MODEL TRAINING. Even contractors end up working with DGX boxes to do all their training.

As of 2023, I was doing the largest Deep learning training runs out of anyone I have known in the industry and I've been in the industry for 20 yeras. The second best groups behind mine were using 4 GPU locally machines that they had to purchase on contract.

There's no way the DoD can train these models themselves, not even close. They are COMPLETELY DEPENDENT ON INDUSTRY. I was the PM for a DARPA program in 2023 and SAME PROBLEM. They had no compute or would rely on university compute if a program had a university partner. YOU HAVE NO IDEA HOW FAR BEHIND THE DOD IS IN THIS SPACE.