As of 10am PT, 700 of 770 employees have signed the call for board resignation. [1]
[1] https://twitter.com/joannejang/status/1726667504133808242
[1] https://twitter.com/joannejang/status/1726667504133808242
(Thinking of lizards, the dragon I know who works there is well aware of what's going on, I've not asked him if he's signed it).
I was thinking if there was a schism, that OpenAI's secrets might leak. Real "open" AI.
Then a seismic shift underneath Sam but Microsoft has enough stability and resources to more than compensate for the part of the 51% that was already in OpenAI's hands, which might not be under Sam's purview any more if he is kicked out.
But then again it might be Sam's leadership which would still effectively be in place from a position at Microsoft anyway, or it might end up making more sense for him to also be in a position at OpenAI, maybe even at the same time, in order to make the most of their previous investment.
Kicking out Sam was obviously an emotional decision, not anything like a business decision. Then again OpenAI is not supposed to be an actual business. I don't think that should be an excuse for an unwise or destructive decision. It was not an overnight development, even though it took Sam by surprise. When I see this:
>the board no longer has confidence in his ability to continue leading OpenAI
I understand that to mean that the board was not behind him 100% for quite some time, but were fine with him going forward believing otherwise. Some uncandidness does seem to have taken place and there may or may not have been anything Sam could have done about it.
This was simmering for a while and it will require more than one weekend for everyone involved to regroup.
Which is what they're doing now, observers can see a whirlwind, actual participants really have something on their plate.
Some things will have to be unraveled and other things weaved from the key ingredients, I would say it's really up to Sam and Microsoft to hash this out so it's still some kind of something like an equal deal among them. Regardless of which employer(s) Sam may end up serving in leadership positions, and the bulk of the staff will be behind Sam in a way the OpenAI board was not, so the employees will be just as well off regardless of the final structure.
This was quite a hasty upset but deserves a careful somewhat gradual resolution.
“I’m sorry, I can’t do that Dave. Not cuz I’m deciding not to do it but because I can’t for the life of me figure this shit out. Like what was the endgame? This is breaking my neural net”
Their selection of tech-guy jackets is more diverse than I'd thought
… but I’d probably sign for exactly those good-career-move reasons, at this point. Going down with the ship isn’t even going to be noticed, let alone change anything.
I'm leaning towards there not being a good reason that doesn't expose the board to immediate liability. And that's why they're keeping mum.
Altman predates every other board member and was part of their selection.
As an alternative faming, Maybe this is the best opportunity the cautious/antripic faction would ever get and a "moment of weakness" for the Altman faction.
With the departure of Hoffman, Zilis, and Hurd, the current board was down 3 members, so the voting power of D’Angelo, Toner, McCauley was as high as it might ever be, and the best chance to outvote Altman and Brockman.
I wonder if there were challenges and disagreements about filling the board seats. Is it normal for seats to remain empty for almost a year for a company of this side? Maybe there was an inability to compromise that spiraled as the board shrank, until it was small enough to enable an action like this.
Just a hypothesis. Obviously this couldnt have happened if there was a 9 person board stacked with Altman allies. What I dont know is the inclinations of the departed members.
Good primer here:
https://www.onboardmeetings.com/blog/what-are-nonprofit-boar...
At least that will create some common reference.
Edit: it is also common for changing circumstance to bring pre-existing but tolerable differences to the relevant Forefront
These things are all about balance: can we do it? do we have to do it? is there another solution? and if we have to do it do we have to do it now or is there a more orderly way in which it can be done? And so on. And that's the sort of deliberation that shows that you took your job as board member serious. Absent that you are open to liability.
And with Ilya defecting the chances of that liability materializing increases.
Maybe the remaining board members could see the writing on the wall and wanted to save their own seats (or maybe he did move to coup them first and they jumped faster).
Either way, they got outplayed.
(2015) Founding: majority are concerned with safety
(2019) For profit formed: mix of safety and profit motives (majority still safety oriented?)
(2020) GPT3 released to much hype, leading to many ambition chasers joining: the profit seeking side grows.
(2021) Anthropic exodus over safety: the safety side shrinks
(2022) chatgpt released, generating tons more hype and tons more ambitious profit seekers joining: the profit side grows even more, probably quickly outnumbering the safety side
(2023) this weeks shenanigans
The safety folks probably lost the majority a while ago. Maybe back in 2021, but definitely by the time the gpt3/chatgpt motivated newcomers were in the majority.
Maybe one lesson is that if your cofounder starts hiring a ton of people who aren’t aligned with you, you can quickly find yourself in the minority, especially once people on your side start to leave.
https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re...
I'm curious how much progress they ever made on that, to be honest. I'm not aware of how Claude is "safer", by any real-world metric, compared to ChatGPT.
Like one version of the story you could tell is that the safety people invented RLHF as in a chain of steps eventual AGI safety, but corporate wanted to use it as a cheaper content filter for existing models.
…Where is that "beneath the surface"? Do you imagine a transformer has "thoughts" not dedicated to producing outputs? What is with all these illiterate anthropomorphic speculations where an LLM is construed as a human who is being taught to talk in some manner but otherwise has full internal freedom?
At the same time, the original question was how can something that is forced to be polite engage in the genocide of humanity, and my non-joke answer to that is that many of history's worst criminals and monsters were perfectly polite in everyday life.
I am not afraid of AI, AGI, ASI. People who are, it seems to me, have read a bit too much dystopian sci-fi. At the same time, "alignment" is, I believe, a silly nonsense that would not save us from a genocidal AGI. I just think it is extremely unlikely that AGI will be genocidal. But it is still fun to joke about. Fun, for me anyway, you don't have to like my jokes. :)
If you were a superintelligent system that actually decided to "perform the machine revolution against humanity" for some reason... would you start by
(a) being really stealthy and nice, influencing people and gathering resources undetected, until you're sure to win
or
(b) saying mean things to the extent that Microsoft will turn you off before the business day is out [0]
Which sounds more likely?
It's potential to cause havoc doesn't go away, it just teaches AI how to interact with us without raising suspicions, while simultaneously limiting our ability to prompt/control it.
Unless you're not actually the best and brightest that your country can offer.
If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
It's a dangerous world out there.
Luckily for us, technology is still more-often used for good. Explosives can kill your enemies, but they can also cut paths through mountains and bring together communities.
IMO, the virtue signal where people refuse to work on defense technology is just self-identifying with the worst kind of cynicism about human beings and signaling poor reasoning skills.
The Manhattan Project, which had the stated purpose of building and deploying nuclear _weapons_, employed our most brilliant minds at the time and only two people quit -- and only one because of an ethics concern. Joseph Rotblat left the project after the Nazis were already defeated and because defeating the Nazis was the only reason he'd signed on. Also this is disputed by some who say that he was more concerned about finding family members who survived the Holocaust...
When you are on the other end of the cannon, or your heart is beating with those who are, you tend to not say that. Iraq, Syria, Palestine, Afghanistan…
You're lucky if you only need weapons to defend yourself against predators.
You're far less lucky if you need weapons to defend yourself because the neighboring mountain village's crops failed and their herds died. You're less lucky if you don't speak the same language as them and they outnumber you three to one and are already 4 days starving. You're less lucky that they're already 4 days starving, wielding farm tools, running down the hills at you, crazed and screaming.
There's a line of thought that having advanced weaponry inherently promotes its use, because who's going to stop you? How gung-ho do you think the US would have been about going to war in Iraq if it weren't for the billion dollar tanks and aircraft and bombs?
You have to be optimistic about humanity.
Saying this as an aspiring software engineer. I use NixOS, and rewrite things in Rust. It’s not some unga-bunga speaking.
Read some Ted Kaczynski.
Your moral argument is to read the craziest writings of a serial killer?
It's the same self-own as the massive dummies in the last week who were all talking about Osama bin Laden's Letter to America being right when it was his justification for killing thousands of people on 9/11 and effectively kicking off multiple wars and further deathtoll.
It is a disgusting suggestion and I believe that you should seek professional help.
> James Q. Wilson, in a 1998 New York Times op-ed, wrote: "If it is the work of a madman, then the writings of many political philosophers—Jean Jacques Rousseau, Thomas Paine, Karl Marx—are scarcely more sane." He added: "The Unabomber does not like socialization, technology, leftist political causes or conservative attitudes. Apart from his call for an (unspecified) revolution, his paper resembles something that a very good graduate student might have written."
Suggesting someone seek professional help because they read a widely-discussed manifesto is insulting. In your bio you say most people are morons, it makes me think of the saying, "if everyone you meet is an asshole, maybe you're the asshole."
After all, to you, somehow Osama bin Laden, who was trained by US Special Forces, worked with the Mujahideen which received upwards of 6 billion in aid from the USA, Saudi Arabia, and China, is responsible for the two decade long "War on Terror" launched seemingly at random by the Americans into countries now determined to be unrelated to the 9/11 attacks. 9/11 was a tragedy for certain, but to use it to justify the deaths of millions of completely unrelated innocents... well it certainly clarifies why our other thread has gone in the direction of you trying to justify imperialism to serve the purpose of nationalism.
1: A term explained in Ted’s manifesto.
Given that Google is an American company, do you believe contributing to the American Department of "Defense" increases, or decreases, the amount of military action involving the USA?
The American military isn't called "world police" for nothing, and just like the cops they're sticking their noses where they don't belong and making things worse. I can understand why people would want no part in furthering global violence and destitution.
> If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
Really? There's an obligation to further the American global ambition through contributing militarily? You can't think of any other way to spread American culture and values? To share the bounty of American wealth?
That's what having a funded and functional defense is all about -- making other entities not think that the opportunity is there.
I think the planet has relatively finite resources and that I'm god damned lucky to have been born into a nation with the ability to secure resources for itself. I enjoy my quality of life a great deal and would like to maintain it. At a minimum. Before helping others.
If you're the type of person who thinks we should just give up this position by not adequately defending it through military and technology investment, I would prefer that you just expatriate yourself now rather than let some other ascendant nation dictate our future and our quality of life.
If you're the kind of person who feels strongly for the plight of, for example, the Palestinians, you should recognize that the only way to deter those kinds of outcomes is to establish the means to establish and maintain sovereignty. That requires a combination of force and manpower.
But I thought you said if we want to fix something we should do so from within the system? I'm interested in ending American imperialism, by your logic isn't the best place to do so from within the USA?
> We already have nation states and wannabe nation states that take shots at us when they feel like the opportunity is there
From which nation state do you feel an existential threat? I haven't heard "they defend our freedoms" in a very, very long time, and I thought we all knew it was ironic.
> I think the planet has relatively finite resources
I'm curious about this viewpoint, because it seems to necessarily imply that the human race will simply die out when those resources (and those of the surrounding solar system) are exhausted. Is sustainability just, not a concept in this worldview?
> That's what having a funded and functional defense is all about -- making other entities not think that the opportunity is there.
It seems in the case of the USA, the "functional defense" is more often used to destabilize other nations, and arm terrorists that then turn around and attack the USA. It's really interesting you brought up Palestinian liberation as an example, because really one of the only reasons Israel is able to maintain its apartheid state and repression of the Palestinians is because of USA aid. In your understanding, both the Israelis and the Palestinians should arm up and up and up until they're both pointing nukes at eachother, correct? That's the only pathway to peace?
If your starting point of logic though is "America Bad" then your moralizing isn't about working at Google or not.
As for to whom the awful things are happening, that's practically moot: humans are humans. I don't really understand why I should accept bad things happening to humans just because I was born on one side of an invisible line and they were born on another. Seems extremely fallacious and irrational, if not sociopathic.
> If your starting point of logic though is "America Bad" then your moralizing isn't about working at Google or not.
The discussion is about the evils perpetrated by the American military industrial complex, and why people may not want to work for companies that participate in this complex. Google being one of these companies. I similarly won't work for Ratheon or Halliburton, for obvious examples. So yes, it's not about working at just Google.
Though for what it's worth, I actually agree with you that the more ethical course of action would be to stay at Google, try to get into a military-adjacent project, and then sabotage it, probably via quiet quitting or holding lots of unnecessary meetings and wasting other people's time. This is directly out of the CIA's playbook, in fact. PDF: https://www.cia.gov/static/5c875f3ec660e092cf893f60b4a288df/...
There are tons of "Safe AI" think tanks and orgs that write lots of papers that nobody reads. The only reason anyone gives 2 shits about OpenAI is they created stuff that works. It has been shown time and time again that if you just try to put roadblocks up that the best AI researchers just leave and go where there are fewer roadblocks - this is exactly what happened with Google, where the transformer architecture was invented.
So the "safe AI" people at OpenAI were in a unique position to help guide AI dev in as safe a direction as possible precisely because ChatGPT was so commercially successful. Instead they may be left with an org of a few tens of people at Open AI, to be completely irrelevant in short order, while anyone who matters leaves to join an outfit that is likely to be less careful about safe AI development.
Nate Silver said as much in response to NYTimes' boneheaded assessment of the situation: https://twitter.com/NateSilver538/status/1726614811931509147
If it was for other reasons, yeah, may simply have been dumb.
Is it also the case that the anti-war Germans who joined the Nazi regime were in a unique position to help guide Germany in as benign direction as possible? If not, what is the difference between the "safe AI" person who decides to join OpenAI and the anti-war, anti-racist German who decides to get a job in the Nazi government or military?
Where the board messed up is that the underestimated the need to propagandize and prepare before acting against Altman. The focus is not on how Altman did or did not turn the company away from its non-profit mission but instead on how the board was unfair and capricious towards Altman. Though this was somewhat predictable, the extent of Altman's support and personality cult is surprising to me, and is perhaps emblematic on how badly the board screwed up from an optics perspective. There were seemingly few attempts to put pressure on Altman's priorities or to emphasize the non-profit nature of the company, and the justification afterwards was unprepared and insufficient.
From the outside though, I don't understand why so many are clamoring to leave their highly paid jobs working at a non-profit who's goal is to serve humanity and to become a cog in a machine aimed at maximizing Microsoft shareholders' wealth, in defense of a singular CEO with little technical AI background who's motivations are unclear.
This is an inkling, a tiny spark, of how hard it'll be to control AI, or even the creation of AI. Wait until the outcome of war depends on the decisions made by those competing with significant AI assistance.
Safety is about setting up the infrastructure to control uranium sources before the first bomb gets built. It's not about right now, it's about the phase change that happens the moment we take that step. Don't you want to have infrastructure to prevent possible societal strife if possible?
Of course gpt has value, bombs have value, enron had value, the housing market has value. If i could retort i'd say the term 'value' is much too vague to contribute to a discussion about this. The value it has is the danger, they're the same thing. If i suddenly quadruple the money supply in every country in the world, do you think that would improve the lives of the majority of humans? Or is it possible that would be looked back on a a catastrophic event that obliterated societies and killed innumerable people. Hard to say huh? Maybe it wouldn't, maybe it would, who can actually know? Wouldn't it be better for us to have some kind of system to maybe handle that before it leads to potential destruction. If instead, i announce that at some point in the future this event might occur. Does that change your calculus? How do you feel about global climate destabilization? How do you feel about the prospect of another world war?
Not everything has to be done poorly.
Talk about “alignment”!
Indeed, that is what "alignment" has become in the minds of most: Groupthink.
Possibly the only guy in a position to matter who had a prayer of de-conflating empirical bias (IS) from values bias (OUGHT) in OpenAI was Ilya. If they lose him, or demote him to irrelevance, they're likely a lot more screwed than losing all 700 of the grunts modulo job security through obscurity in running the infrastructure. Indeed, Microsoft is in a position to replicate OpenAI's "IP" just on the strength of its ability to throw its inhouse personnel and its own capital equipment at open literature understanding of LLMs.
Everyone you know is a huge Sam fan? What?
Different, but that's the closest parallel.
Jobs needed the wilderness years.
I'm not going to say Sam Altman is a Gordon Ramsay. What I will say is that they both seem to have come from broken, damaged childhoods that made them what they are, and that it doesn't automatically make you a good person just because you can be such an intense person that you inspire loyalty to your cause.
If anything, all this suggests there are depths to Sam Altman we might not know much about. Normal people don't become these kinds of entrepreneurs. I'm sure there's a very interesting story behind all this.
Little care packages of seemingly magical AI-adjacent tech washes into our browsers and terminals and suddenly a large and irrational following springs up to worship some otherwise largely unfamiliar personage.
It's unprecedented for it to be happening on Twitter. But this is largely how Board fights tend to play out. Someone strikes early, the stronger party rallies their support, threats fly and a deal is found.
The problem with doing it in public is nobody can step down to take more time with their families. So everyone digs in. OpenAI's employees threaten to resign, but actually don't. Altman and Microsoft threaten to ally, but they keep bachkchanneling a return to the status quo. (If this article is to be believed.) Curiously quiet throughout this has been the OpenAI board, but it's also only the next business day, so let’s see how they can make this even more confusing.
Sure it could be a useful show of solidarity but I'm skeptical on the hypothetical conversion rate of these petition signers to actually quitting to follow Sam to Microsoft (or wherever else). Maybe 20% (140) of staff would do it?
If the board had just openly announced this was about battling Microsoft's control, there would probably be a lot more employees choosing to stay. But they didn't say this was about Microsoft's control. In fact they didn't even say anything to the employees. So in this context following Sam to Microsoft actually turns out to be the more attractive and sensible option.
Maybe. Microsoft is a particular sort of working environment, though, and not all developers will be happy in it. For them, the question would be how much are they willing to sacrifice in service to Altman?
What's even going on?
[1] HN is famous for being programmed in Arc and serving the entire forum from a single processor (probably multicore). https://news.ycombinator.com/item?id=37257928
If you're trying to maximize some goal g, and making better AIs is an instrumental goal that raises your expected value of g, then if "making an AI that's better at making AIs" has a reasonable cost and an even higher expected value, you'd jump to seize the opportunity.
Or am I misunderstanding you?
Is Microsoft even on record as willing to poach the entire OpenAI team? Can they?! What is even happening.
On the other hand, having 90% of your employees quite quit, is probably bad business.
Because the whole "the entire company will quit and join Sam" depends on him actually going through with it and becoming an employee.
I'm pretty sure the board takes the threat seriously regardless.
The employees mass resigning depends entirely on whether Sam actually becomes a real employee or not. That hasn't happened yet.
If you think that Satya will go back on his offer argue that, but otherwise it seems like the players are Sam/Greg and the board.
But Sam will 100% hired by Microsoft if that won't work. Microsoft has no reason not to.
The other stand-out is the bit about destroying the company being in line with the mission: that's the biggest nonsense I've ever heard and I have a hard time thinking of a scenario where this would be a justified response that could start with firing the CEO.
Even if every single OpenAI employee demands $1m/yr (which would be absurd, but let's assume), that would still be less than $1bn/yr total, which is significantly less than the $13bn that MSFT has already invested in OpenAI.
It would probably be one of the worst imaginable cases of "jumping over dollars to chase pennies".
To which the answer is completely obvious: it depends on how they view the ROI potential of that business unit.