zucker42

Convergent instrumental goals[1] and the orthogonality thesis[2], among other reasons.

[1] https://youtu.be/ZeecOKBus3Q?si=cYJUaxjIJPIbubRL

[2] https://youtu.be/hEUO6pjwFOo?si=DXVosLh6YTsMkKOx

Ilya's talking about type 4.

If I had to bet money on it, researchers at top labs have already tried applying search to existing models. The idea to do so is pretty obvious. I don't think it's the one key insight to achieve AGI as the author claims.

The main thing I'd be worried about in the short term is models making accessible the information to synthesize a pandemic capable virus.

Exactly. If the board had the reasons in this article ready to go when they fired Altman, things may have gone much differently.

I agree that the board overestimated their own power, and underestimated the need to get their business partners on board with Altman's firing. However, I think calling Microsoft the board's boss implies that Microsoft has some sort of moral or legal high ground.

One way this debacle has been portrayed in the media is "an unaccountable board tried to destroy a profitable company". I think a more accurate portrayal is "Sam Altman and Microsoft worked together to deemphasize the company's scientific and humanitarian goals and emphasize building successful and profitable products". It's sort of depressing how Microsoft was able to "capture" a nonprofit.

Major investors aren't the board's boss in this case because OpenAI is a nonprofit. Microsoft is incentivized to act against the goals nonprofit's charter (and in fact they did in this case).

That was definitely part of it.

> "it's Nintendo's own fault for not releasing their games on PC and Android"

I understand if you disagree with this argument (I don't know if I'd endorse it wholeheartedly), but I don't see how it is asinine. The wider context here is that there are people who believe in "general computing", i.e. the idea that users should retain full control over computing devices and software they buy.

The philosophy of companies of companies like Nintendo is that when you buy a game from them, you are buying the right to play the game in exactly the way they want. If they could make modding illegal, they would (it is in Japan, to my understanding). This runs counter to the idea of general computing, where you should be to play a game you buy in any way you want, including modding, or playing on a different device, etc.

So to return to the point, the argument is that by refusing to sell their games to users of other platforms (as well as their other actions), Nintendo is working against the goal of general computing, and therefore it's not worth feeling sorry for them when people pirate the game. Phrased differently, people who don't own a Switch have no way to play the game besides pirating, and people should be able to buy games separately from devices.

Another fundamental argument here is that hardware/software walled gardens which are enforced by anti-circumvention and copyright laws are basically anti-competitive monopolies.

A more accurate rewording of Nintendo's statement (keeping their point of view):

> A video emulator is piece of software that allows a user to run a video game on a platform different from that for which is was written. Many people use emulators to play unlawfully obtained pirated games.

Wording something as definition implies that the statement both contains no falsehoods and is complete.

Saying that the purpose of an emulator is to play pirated games is not true. I use an emulator to play Super Smash Bros Melee on my computer, with the ROM obtained from the disk I own. People have modded the Super Smash Bros Melee game to allow for online play, something that would be impossible without an emulator. People used emulators to find a bug in Ocarina of Time that would allow them to beat the game (played on original hardware) faster.

But the statement is written to confuse a non-technical fact finder into thinking that the only purpose of an emulator is to play a video game without paying for it, or to help others to do so. And it's done so because that Nintendo doesn't believe that you own the game you buy from it.

Hammers are used for theft, as well. Even if Nintendo's business would benefit from emulators not existing[1], it doesn't mean we should ban emulators (or create laws which allow multinationals to sue open source emulator projects out of existence).

[1] which is not necessarily true

Two paragraphs into the legal complaint[1], Nintendo says such BS as:

> A video game emulator is a piece of software that allows users to unlawfully play pirated video games that were published only for a specific console on a general-purpose computing device

[1] https://www.scribd.com/document/709016504/Nintendo-of-Americ...

The yearly odds of a "Bond villian" becoming U.S. president seem a lot higher than the risk of a catastrophic asteroid to me.

> Who is saying that?

There is a quote in the top level comment of this thread that says that.

> It’s untenable that there’s unsanctioned client software for a messaging platform for which privacy and security are a primary feature.

I think plenty of people (myself among them), have used Nouveau without issues.

I'd prefer to have the option to use my current Nvidia card with open source drivers, or to purchase a new one (I care more about good open source drivers now than I did when I bought my current card, in large part because of Wayland). Plus, Nvidia has a virtual monopoly on neural network acceleration because of CUDA, and that seems like it will last at least a few more years, so for some people it's not as simple as "buy AMD".

Second, my understanding of Linux's history is that people creating open source drivers for hardware is responsible for its success. The problem of bootstrapping Linux hardware compatibility was solved in part by people writing open-source drivers when companies didn't support Linux. Community support has been necessary for the creation and upstreaming of many previous drivers, not least among them AMDGpu.

Also, in this case, a large portion of the work for Nouveau is funded by Red Hat. I believe they business reasons for wanting a well-functioning open source Nvidia driver.

Automated software writing from a codebase and a requirement description, with the model asking clarifying questions where necessary, would be an immensely valuable application. We're quite far away from that, but GPT-4 is already rather helpful for some coding tasks.

What is your definition of reasoning? In my mind, GPT-4 has some nascent reasoning abilities.

One thing we've learned over the past few days is that Toner had remarkably little control over OpenAI's actions. If a non-profit's board can't fire the CEO, they have no way to influence the organization.

Ilya and Adam switched because they lost, and their goal wasn't to nuke OpenAI, simply to remove Sam. Helen and Tasha had the votes to prevent Sam Altman from returning as CEO, but not the votes to prevent the employees from fleeing to Microsoft, which Helen and Tasha see as the worst possible outcome.

Do you have a source for either of these things? The only thing I heard about Saudi investors was related to the (presumably separate) chip startup.

Classic CEO worship. There's no reason to believe that he's more important than many of the other key figures in the company.

The "leader of the coup" has changed each day. First it was Ilya, then D'Angelo because of Poe, then Toner because he criticized her paper.

Who, it's worth mentioning, got a seat because Open Philanthropy donated $30 million to OpenAI early in its creation.

I think you're taking the intuition from a for-profit company and wrongly applying it to a non-profit company. When a board member criticizes a for-profit company, that's bad because the goal of the company is make a profit, and bad PR is bad for profit. A board member criticizing a non-profit doesn't have the same direct connection to a result opposite of the goals of the company. And if you actually read the page, it's an extremely mild criticism of OpenAI's decisions.

This situation is simultaneously "reckless board makes ill-considered mistake to suddenly fire CEO with insufficient justification" and "non-profit CEO slowly changes the culture of a non-profit to turn it into profit-seeking enterprise that he has a large amount of control over".

I don't think that the firing of Sam Altman was due to commercialization of GPT-3 and GPT-4, and I don't think there's been any reliable information that confirms that. There's been multiple statements that at least partially contradict that, including:

Emmett Shear: "Before I took the job, I checked on the reasoning behind the change. The board did not remove Sam over any specific disagreement on safety, their reasoning was completely different from that. I'm not crazy enough to take this job without board support for commercializing our awesome models." [1]

Brad Lightcap: We can say definitively that the board’s decision was not made in response to malfeasance or anything related to our financial, business, safety, or security/privacy practices. This was a breakdown in communication between Sam and the board.” [2]

Also, I believe Adam D'Angelo made some pro-commercialization comments on Twitter earlier this year.

[1] https://twitter.com/eshear/status/1726526112019382275

[2] https://fortune.com/2023/11/19/sam-altman-was-fundraising-fo...

Many countries have immigration systems that are similarly bad the U.S.'s.

The main mistake the board made was tactical, not philosophical. From the outside, it's seems likely that Altman was running OpenAI so as to maximize the value of the for-profit entity, rather than achieve the non-profit's goals, if only because that's what he's used to doing as a tech entrepreneur. Looking at OpenAI from the outside, can you honestly say that they are acting like a non-profit in the slightest? It's perfectly believable that Altman was not working to further the non-profit's mission.

Where the board messed up is that the underestimated the need to propagandize and prepare before acting against Altman. The focus is not on how Altman did or did not turn the company away from its non-profit mission but instead on how the board was unfair and capricious towards Altman. Though this was somewhat predictable, the extent of Altman's support and personality cult is surprising to me, and is perhaps emblematic on how badly the board screwed up from an optics perspective. There were seemingly few attempts to put pressure on Altman's priorities or to emphasize the non-profit nature of the company, and the justification afterwards was unprepared and insufficient.

From the outside though, I don't understand why so many are clamoring to leave their highly paid jobs working at a non-profit who's goal is to serve humanity and to become a cog in a machine aimed at maximizing Microsoft shareholders' wealth, in defense of a singular CEO with little technical AI background who's motivations are unclear.

There's a lot of speculation that Sam Altman will return to OpenAI. There are publicly disgruntled employees and high level employees resigning. The executives of the company are reportedly meeting with the board about reinstating Sam Altman. I definitely don't think that's ideal for the board.

The board who fired Sam Altman is the board of OpenAI Inc., the nonprofit. Microsoft has no ownership stake in OpenAI Inc.