The Inside Story of Microsoft's Partnership with OpenAI
newyorker.com
newyorker.com
> ... [Altman's] tactical skills were so feared that, when four members of the board—Toner, D’Angelo, Sutskever, and Tasha McCauley—began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board,” the person familiar with those discussions said.
> The person familiar with [Altman's] perspective said that he and the board had engaged in “very normal and healthy boardroom debate,” but that some board members were unversed in business norms and daunted by their responsibilities.
> [Re: the board's silence after firing Altman] Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints.
This seems to be the most concrete information about what he did since it happened. Sounds like he lied for years and justified it as "business norms."
Maybe she thought she's doing the right thing, I'm sure all those people think they're doing their earnest version of "the right thing". Yet the company she praised, Anthropic, is clearly dysfunctional, with users complaining all over the Internet that Claude refuses to perform basic tasks out of misguided "safety concerns" ("I can't help you kill Linux processes, as this is unethical"), making their subscriptions useless. That's the Helen Toner model.
See, people who deal with security often like to put the spotlight on themselves by proclaiming grave danger is imminent and then position themselves as the experts who can save us from this danger. This is such a common pattern and sometimes it's not even consciously done, but extremely toxic regardless. Everyone wants to be important and regarded, so they lean on behaviors that achieve that outcome.
Sam Altman is your typical startup entrepreneur sleezeball. And yes, here he was trying to manufacture consent within a board that was largely complacent through "divide and conquer". The details of his behavior can be nauseating, but still... I can't deny on the total he did make countless correct leadership decisions at OpenAI, and getting Helen Toner off the board... for all we know so far... feels like one of those correct leadership decisions. So I can't feel too bad about it.
The only thing I feel bad about it is Ilya Sutskever. I hope he and Altman can find a productive and dignified way to continue their partnership.
An inflammatory post about the dangers of AI increases awareness and emotional engagement. Especially paired with a calm and steady press release.
It's the same pattern as an abusive relationship - cause the pain, then alleviate it.
I'm not saying its an explicit manipulation, just a unconscious tactic that opens up when people really care about the mission.
This sounds a lot different from the narrative that it was a grass roots show of support for Sam Altman.
If the CEO, the rest of the execs, and the investors in the for-profit were pushing employees to sign, doing so wasn’t a sign of loyalty to Altman at all.
The word Microsoft used to be borderline toxic.
Its thesis is that VSCode is meant to leverage open source and popularity to increase closed source vendor lock-in to Microsoft. It cites recent examples of VSCode extensions being closed source, so that while VSCode seems open, the critical portions are not. It's worth reading the full article.
"Well, sure, but [the board not trusting Altman] is a fight about AI safety. It’s just a metaphorical fight about AI safety. I am sorry, I have made this joke before, but events keep sharpening it. The OpenAI board looked at Sam Altman and thought 'this guy is smarter than us, he can outmaneuver us in a pinch, and it makes us nervous. He’s done nothing wrong so far, but we can’t be sure what he’ll do next as his capabilities expand. We do not fully trust him, we cannot fully control him, and we do not have a model of how his mind works that we fully understand. Therefore we have to shut him down before he grows too powerful.'
I’m sorry! That is exactly the AI misalignment worry! If you spend your time managing AIs that are growing exponentially smarter, you might worry about losing control of them, and if you spend your time managing Sam Altman you might worry about losing control of him, and if you spend your time managing both of them you might get confused about which is which. Maybe Sam Altman will turn the old board members into paper clips."
(Link: https://www.bloomberg.com/opinion/articles/2023-11-29/the-ro...)
I don't know the guy but frankly from reading a lot of stuff on here posted by people that have experience with him I am not inclined to trust him at all with anything and would watch him like a hawk. Trust is critical and he does not inspire trust.
This is the part that his sycophants shout over so nobody can hear it. The Board™ didn't trust Altman. He isn't a martyr, just another startup success rotting on the vine.
"For Runciman, we’ll be lucky if we can coax into alignment the machines we’ve made of ourselves, never mind the ones made of silicon."
An AGI, or especially an ASI, would presumably avoid such easily-detectable mistakes. Things just start happening and seemingly disconnected events steer reality in a particular direction, and there is no way to concretely link them or discern their true provenance.
This is already exactly the problem with the most sophisticated disinformation campaigns run by the FSB/IRA/etc. It appears random, grassroots, and uncoordinated, and is difficult or impossible to discern its true provenance. But it all operates across a shared a set of objectives - increase social divisions, decrease trust in democracy and democratic institutions, and/or increase general irrationality and emotional response among the public.
If humans can already accomplish that, imagine what an ASI could do.
I think he's very obviously overconfident and sloppy.
This is the essence of playing poker- the bluffs need to be called for it to be a bad bluff.
Unless somebody can make some sort of extremely convincing argument that by deploying this tech we are taking an existential risk, rather than purely banal ones, it doesn't seem reasonble to prevent deployment.
A better example is perhaps pharmaceuticals, many of which we do not actually understand the mechanisms of action. But note those go through extremely rigorous testing for exactly that reason.
Similar issues with molecular biology- we knew how to clone a gene from one organism to another, but that doesn't mean we really knew all the things going on (side effects) or what the large-scale implications are (hence the Asilomar agreement).
Even cars- while engines were understood mechanistically, it took quite some time for people to appreciate why automobile safety glass was necessary.
See the experiences learned during testing nuclear weapons- we tested them because we didn't understand them mechanistcally, at least not fully enough to predict many effects.
We have effectively no mechanistic understanding of frontier ML systems, in the sense that we have no idea how they do what they do and could not e.g. write human-readable code to perform comparable tasks, nor can we predict ahead of time what capabilities such systems will have when they're trained (being able to predict e.g. log-loss is _not_ the same as being able to predict specific capabilities).
But my god, in what world should the future of “AI safety” be in the hands of a small board of directors that works in a completely opaque manner and with uncertain motivations or capabilities?
Surely humanity’s fate is not meant to be guarded by the Quora dude.
Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought,” the person familiar with the board’s discussions told me. “Things like that had been happening for years.”This concern (that a profit-maximizing CEO like Sam is not the type of person who should decide the ethical fate of humanity) was and is the whole point of having a non-profit as the owner of the for-profit tech company, so that ethically-driven people who value human well-being over growth can be in charge of the non-profit board and exercise final say over the AI's ethical constraints.
The fact that Microsoft now has a board observer seat seems highly corrupting, and contrary to the board's fiduciary purpose. This is discussed here: https://news.ycombinator.com/item?id=38471990 Basically, Microsoft now has an enormous, and indecent, insider information advantage (allowing them to conduct software development industry market front-running, which would be illegal but it will be impossible to prove that the source of their ideas is the "forbidden fruit" of what they hear in board meetings). Details may be found in that link.
While I'm at it, most of the references to the paper co-authored by Toner missed the two pages of praise for OpenAI before the single paragraph with the somewhat negative comparison to Anthropic.
Excerpts:
"...Some members of the OpenAI board had found Altman an unnervingly slippery operator. For example, earlier this fall he’d confronted one member, Helen Toner, a director at the Center for Security and Emerging Technology, at Georgetown University, for co-writing a paper that seemingly criticized OpenAI for “stoking the flames of AI hype.” Toner had defended herself (though she later apologized to the board for not anticipating how the paper might be perceived). Altman began approaching other board members, individually, about replacing her. When these members compared notes about the conversations, some felt that Altman had misrepresented them as supporting Toner’s removal. “He’d play them off against each other by lying about what other people thought”, the person familiar with the board’s discussions told me. “Things like that had been happening for years.” (A person familiar with Altman’s perspective said that he acknowledges having been “ham-fisted in the way he tried to get a board member removed”, but that he hadn’t attempted to manipulate the board.)
...His tactical skills were so feared that, when 4 members of the board---Toner, D’Angelo, Sutskever, and Tasha McCauley---began discussing his removal, they were determined to guarantee that he would be caught by surprise. “It was clear that, as soon as Sam knew, he’d do anything he could to undermine the board”, the person familiar with those discussions said...Two people familiar with the board’s thinking say that the members felt bound to silence by confidentiality constraints...But whenever anyone asked for examples of Altman not being “consistently candid in his communications”, as the board had initially complained, its members kept mum, refusing even to cite Altman’s campaign against Toner.
...The dismissed board members, meanwhile, insist that their actions were wise. “There will be a full and independent investigation, and rather than putting a bunch of Sam’s cronies on the board we ended up with new people who can stand up to him”, the person familiar with the board’s discussions told me. “Sam is very powerful, he’s persuasive, he’s good at getting his way, and now he’s on notice that people are watching.” Toner told me, “The board’s focus throughout was to fulfill our obligation to OpenAI’s mission.” (Altman has told others that he welcomes the investigation---in part to help him understand why this drama occurred, and what he could have done differently to prevent it.)
Some A.I. watchdogs aren’t particularly comfortable with the outcome. Margaret Mitchell, the chief ethics scientist at Hugging Face, an open-source A.I. platform, told me, “The board was literally doing its job when it fired Sam. His return will have a chilling effect. We’re going to see a lot less of people speaking out within their companies, because they’ll think they’ll get fired---and the people at the top will be even more unaccountable.”
Altman, for his part, is ready to discuss other things. “I think we just move on to good governance and good board members and we’ll do this independent review, which I’m super excited about”, he told me. “I just want everybody to move on here and be happy. And we’ll get back to work on the mission”."Wait, what? Apologise for doing your job or did the board fail to read Helens current job description? Or is Helens position on the board of openai meant to prevent criticism of openai?
Second, the 'criticism' here is laughably weak and is barely even a criticism at all (I actually read it as a criticism of everyone else but OA); I strongly encourage anyone who thinks that this might justify firing someone (especially without, apparently, a replacement who would be acceptable to the safety faction) to actually read the supposedly unforgivable page of the paper: https://cset.georgetown.edu/wp-content/uploads/CSET-Decoding...
that's one of the fundamental jobs of the board: to be critical and to guide direction, especially if the direction is different from what is happening at the moment.
and yes, multiple Ford board members were/are extremely critical of Ford's approach to cars and are very public about it. That's why multiple CEO's were sacked, and even members of the Ford family are not spared from criticism from the Ford Board.
The board of even a nonprofit can have motivations. Your original comparison is still false and unproven. Sorry.
https://www.macrotrends.net/stocks/charts/F/ford-motor/profi...
Lesson here is… the next time a startup CEO gets fired they should try the social media spam gambit…? Depends on how much $ is on the line but worked at least once!
Really bizarre that they didn't do that.
Compare the "triumphant return" letter he wrote (https://openai.com/blog/sam-altman-returns-as-ceo-openai-has...), versus the "we're jettisoning Sam because he's not candid" letter: https://openai.com/blog/openai-announces-leadership-transiti...
The board's letter is a big "WTF" that led to their downfall. Sam's is basically "I'm back and bigger than ever, thanks to all the minions who helped, now let's get back to implementing the singularity".
Psychopath.
This is funny language, it's been adopted into the corporate bullshit buzz phrase guide thanks to Altman.
"As enticing as Plan C initially seemed, Microsoft executives have since concluded that the current situation is the best possible outcome. Moving OpenAI’s staff into Microsoft could have led to costly and time-wasting litigation, in addition to possible government intervention. Under the new framework, Microsoft has gained a nonvoting board seat at OpenAI, giving it greater influence without attracting regulatory scrutiny."
If they were involved in anyway, wouldn't they actually know this was about to happen?
Your version of events requires us to assume all the board members are lying and Altman is lying and Microsoft is Lying and you and you alone know the truth.
EDIT after reading some of your other comments, its clear you aren't operating based on facts, but just some made up fantasy that you want to believe:(
I regret trying to engage in meaningful dialog with someone who isn't trying to argue in good faith:(
Really?
How did you reach that conclusion?
All the evidence points to MS being blindsided
When a company of sheisters repeatedly sheists their way through history then assume interactions by them will generally be sheisty. I'd say this is some sort of reverse Hanlon's razor.
As a sailor of yore manning a merchant ship, when approached by a frigate flying skull and crossbones I suppose you'd say "but captain we cannot know their true intentions perhaps they wish for some adhoc trade! It would be premature and I dare say prejudiced to assume otherwise!"
I hate Microsoft for their demonstrated shady practices, I do not them for fictional sins they might, or probably not, committed
That sounds like the kind of name an AI incompetently trying to disguise itself would use.
i agree sam is too smart for them, they need to replace the board
I don't see how you can build an $80 billion operation, with a board full of blunderers. I get that it was those blunderers AND sam, until about a year ago; and I get that a year earlier still, they had several experienced businessmen on the (then larger) board.
None of the "blunderers" has spoken yet, AFAIK; I doubt they've taken the cloth and become Trappists, so I assume they're keeping their powder dry, and waiting to see how the situation shakes out.
In many places the role of the board is just to fire / hire the CEO. In some orgs they have more roles: getting financing, customers, audit, agreeing on certain things (e.g. investments, or contracts). Sometimes the board also has more prerogatives.
But in general the role is a more ceremonial role that once per year chooses the audit firm (one of the big 4 companies.. so not very hard), maybe makes some obligatory resolutions, like confirming the audit result. Their biggest power is to remove the CEO and choose the CEO's compensation.
Getting a seat on a board of a public company is like free money. Most boards dont meet often. Typical boards in companies represent the shareholders (aka remove the CEO if they want to do something crazy), while in non profits they are usually there to find financing or networking.
So yes, you can get very far with a bad board, because most boards are quite inactive. The members read some documents, maybe sometimes request some data - and that's all.
Also you are wrong.
Yes, I guess I knew that about boards, subliminally.
> So yes, you can get very far with a bad board
That I didn't know. I'm inclined to believe it, and it seems at first glance to clear-up in my mind quite a few things about government in general that keep surprising me.