OpenAI Is a Strange Nonprofit
bloomberg.com
bloomberg.com
From the board’s perspective ... will have a staffing problem ... those staff will probably be more enthusiastic about AI, generally, than the mission calls for.
From the staff’s perspective, the board is a bunch of outsiders ... driven by an abstract sense of mission. Which kind of is the job of a nonprofit board, but which will reasonably annoy the staff.
Yesterday virtually all of OpenAI’s staff signed an open letter to the board ... the letter claims that the board “informed the leadership team that allowing the company to be destroyed ‘would be consistent with the mission.’” Yes! I mean, the board might be wrong about the facts, but in principle it is absolutely possible that destroying OpenAI’s business would be consistent with its mission. If you have built an unsafe AI, you delete the code and burn down the building. The mission is conditional — build AGI if it is safe — and if the condition is not satisfied then you go ahead and destroy all of the work. That is the board’s job. It’s the board’s job because it can’t be the staff’s job, because the staff is there to do the work, and will be too conflicted to destroy it. The board is there to supervise the mission.
Assuming AGI is possible at all, this is why something like OpenAI should never have even been started if the concern was anything like ensuring safety or even being a net benefit to the world.
Fortunately, I think AGI isn't possible (at least not within a short enough timeline to matter). But I do think there's been nothing but recklessness in terms of what has already been accomplished. In the short term, uninvolved others are inevitably going to suffer. I have a problem with any group of people deciding that others are expendable.
Could you elaborate on this? Specifically which accomplishment(s) you consider to be reckless.
"Fortunately"? That's a strange way to phrase that. Most people in 1928 thought that the repeat of the Great War isn't possible but it happened anyhow in just 10 years. Heck, nobody thought WWI, in the way it actually happened, could be possible! A world-wide war, in a globalized (yes, it was globalized about as much as in the 1990s) economy? A war where people would just keep dying on the field from both sides without any territorial advances, and keep doing this for years, and the involved countries mostly just waiting to see who will first run out of the resources (spoiler: Germany and Austria ran out first, because the US got involved), material and human? Nah, that's just un-scientific fantasy only some silly socialists like Wells could come up with.
But it doesn't need to get to that end point - if the board thought that the decisions the CEO had made or likely to make in the future would not lead to the core goal of "building AGI that is safe and benefits all of humanity" - because, for example, future work would be dominated by links to Microsoft, or he would be distracted by building other for profit companies that built on OpenAI work, they they would also be right to sack him even if it meant destroying the profitability/growth of the company.
I think the structure of OpenAI was a hideous idea - but I can't help but feel the board may have acted according to the rules they'd been given.
It’s even directly contrary to the statements from the board and the new CEO, and there is no reason to believe they would lie and keep it hidden if this was in fact their reasoning.