Hacker News Reader

Comment by sanderjd | Hacker News Reader

Well, their big mistake was being unwilling to be clear and explicit about this, but as I read it, the board's problem with him was that he wasn't actually acting as the executive of the non-profit that he was meant to be the executive of, but rather was acting entirely in the interests of a for-profit subsidiary of it (and in his own interests), which were in conflict with the non-profit's charter.

I think where they really screwed up was in being unwilling or unable to argue this case.

View on HN

It's just so strange. This is such a clearly justifiable reason that the fact that they didn't argue it... or argue.. anything, makes me very suspicious that it is correct.

They were either scared of being sued for defamation or unwilling to divulge existential company secrets. Or both.

Doesn't really make sense to be unwilling to divulge company secrets if you're willing to gut the company for this hill.

They weren't willing to gut the company. That's why Sam is back as CEO.

It sounded like they would if they could (for instance trying to sell to Anthropic or instating a "slow it way down" CEO), but they even failed at that. Not an effective board at all.

>for instance trying to sell to Anthropic or instating a "slow it way down" CEO

I wouldn't put these in the same category as "90% of staff leaves for Microsoft".

In any case, let's not confuse unsuccessful with incompetent. (Or incompetent with immoral, for that matter.)

They were willing but failed, and mostly on account of not doing enough prepwork.

It's remarkable that the old board is the side characterized as willing to blow up the company, since it was Altman's side who threatened to blow it up. All the old board really did was fire Altman and remove Brockman from the board.

Exactly. It sounds like those board members themselves were acting in the interest of profit instead of the "benefit all humanity" mission stuff, no different than Altman. If anything then, the only difference between the two groups is one of time horizon. Altman wants to make money from the technology now. The board wants to wait until it's advanced enough to take over the world, and then take over the world with it. For the world's benefit, of course.

I think "unwilling to divulge company secrets" is the best explanation here.

We know that OpenAI does a staged release for their models with pre-release red-teaming.

Helen says the key issue was the board struggling to "effectively supervise the company": https://nitter.net/hlntnr/status/1730034017435586920#m

Here's Nathan Labenz on how sketchy the red-teaming process for GPT4 was. Nathan states that OpenAI shut him out soon after he reached out to the board to let them know that GPT4 was a big deal and the board should be paying attention: https://nitter.net/labenz/status/1727327424244023482#m [Based on the thread it seems like he reached out to people outside OpenAI in a way which could have violated a confidentiality agreement -- that could account for the shutout]

My suspicion is that there was a low-level power struggle ongoing on the board for some time, but the straw that broke the camel's back was something like Nathan describes in his thread. To be honest I don't understand why his thread is getting so little play. It seems like a key piece of the puzzle.

In any case, I don't think it would've been right for Helen to say publicly that "we hear GPT-5 is lit but Sam isn't letting us play with it", since "GPT-5 is lit" would be considered confidential information that she shouldn't unilaterally reveal.

So what is Nathan Labenz saying? That GPT-4 is dangerous somehow? It will get many people out of jobs? MS Office got all the typists out of jobs. OCR and Medical Software got all the medical transcriptionists out of jobs. And they created a lot more jobs in the process. GPT-4 is a very powerful tool. It has not a whiff of AGI in it. The whole AGI "scare" seems to be extremely political.

Nathan says the initial version of GPT-4 he red-teamed was "totally amoral" and it was happy to plan assassinations for him: https://nitter.net/labenz/status/1727327464328954121#m

Reducing the cost of medical transcription to ~$0 is one thing. Reducing the cost of assassination to ~$0 is quite another.

This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.

I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility.

In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.

Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.

Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.

What kind of background check do you think exists when buying a fighter jet?

It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.

Sure, but idiots are a thing and the intersection of the sets of libertarians who may believe that and idiots is hopefully empty but it may not be so such outsized power is best dealt with through a chain of command and accountability of sorts.

Amoral literally means "lacking a moral sense; unconcerned with the rightness or wrongness of something." Generally this is considered a problem if you are designing something that might influence the way people act.

We don't want immoral output even for immoral input.

(We do disagree about what constitutes "immoral", which makes this much harder).

We absolutely do, though, if we want those things to e.g. write books and scripts in which characters behave immorally.

Then we should stop teaching Therac-25 incident to developers and remove envelope protection from planes and safety checks from nuclear reactors.

Because, users should just input the moral inputs to these things. These are inanimate objects too.

Oh, while we're at it, we should also remove battery charge controllers. Just do the moral and civic thing and unplug when your device charges.

In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.

> In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.

Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator.

An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems.

Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.

Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.

> Why would anyone give control of air traffic or weapons to AI?

Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept.

> a chatbot.

Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits.

> I can't see it happening.

Because, it already happened.

Turkey is claimed to use completely autonomous drones in a war [1].

South Korea has autonomous sentry guns which defend DMZ [2].

[0]: https://www.imdb.com/title/tt0086567/

[1]: https://www.popularmechanics.com/military/weapons/a36559508/...

[2]: https://en.wikipedia.org/wiki/SGR-A1

We give hall passes to more than AI. We give passes to humans. We could have a detailed discussion of how to blow up the U.S. Capitol building during the State of the Union address. It is allowed to be a best selling novel or movie. But we freak out if an AI joins the discussion?

Yes, of course. But that is precisely what people mean when they say that the problem isn't AI, it's people using AI nefariously or negligently.

"The problem isn't that anyone can buy an F16. The problem is that some people use their F16 to conduct airstrikes nefariously or negligently."

You persist in using highly misleading analogies. A military F-16 comes with missiles and other things that are in and of themselves highly destructive, and can be activated at a push of a button. An LLM does not - you'd have to acquire something else capable of killing people first, and wire the LLM into it. The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.

Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.

>The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.

The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier.

>Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?"

https://nitter.net/ESYudkowsky/status/1660399502266871809#m

The point here is that is giving the user detailed knowledge on how to harm others. This is way different than a gun where you are doing the how (aiming and pulling the trigger).

The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.

Thing is, this detailed knowledge is already available and much easier to acquire. There are literally books on Amazon that explain how to jury-rig firearms and bombs etc. Just to give one example: https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/....

When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it.

I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.

The cost of planning an assassination is not the same thing as the cost (and risk) of carrying out an assassination, what a stupid take.

A would be assassin would obviously ask the algorithm to suggest a low risk and cost way of assassinating.

Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone.

And everything LLM can come up with will be exactly the same information you can find in any fiction detective book or TV series about crime. Yeah very very dumb criminal can certainly benefit from it, but he can as well go on 4chan and ask about assassination there. Or on some detective book discussion club or forum.

>And everything LLM can come up with will be exactly the same information you can find in any fiction detective book or TV series about crime.

As Nathan states:

>And further, I argued that the Red Team project that I participated in did not suggest that they were on-track to achieve the level of control needed

>Without safety advances, I warned that the next generation of models might very well be too dangerous to release

Seems like each generation of models is getting more capable of thinking, beyond just regurgitating.

I dont disagree with his points, but you completely miss the point of my post. People dont need an AI advise to commit crime and kill others. Nonestly humans they're pretty good at it using technology of 1941.

You don't have bunch of cold blood killers going around not just because police is so good and killers are dumb and need AI help. It's because you live in functioning state where society have enough resources so people happy enough to instead go and kill each other in Counter Strike or Fortnite.

I totally agree that AGI could be a dangerous tech, but it's will require autonomity where it can manipulate real world. So far GPT with API access is very far from that point.

If you have ChatGPT API access you can have it write code and bridge that to other external APIs. Without some guard rails an AI is like a toddler with a loaded gun. They don't understand the context of their actions. They can produce dangerous output if asked for it but also if asked for something else entirely.

The danger also doesn't need to be an AI generating code to hack the Gibson. It could also be things like "how do I manipulate someone to do something". Asking an AI for a marketing campaign isn't necessarily amoral. Asking it how to best harass someone into committing self-harm is.

> Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone

Most of us don't want to.

Most of those who do, don't know enough to actually do it.

Sometimes such people get into power, and they use new inventions like the then-new-pesticide Zyklon B to industrialise killing.

Last year an AI found 40k novel chemical agents, and because they're novel, the agencies that would normally stop bad actors from getting dangerous substances, would generally not notice the problem.

LLMs can read research papers and write code. A sufficiently capable LLM can recreate that chemical discovery AI.

The only reasons I'm even willing to list this chain, is that the researchers behind that chemical AI have spent most of the intervening time making those agencies aware of the situation, and I expect the agencies to be ready before a future LLM reaches the threshold for reproducing that work.

Everything you say does make sense, except those people who able to get equipment to produce those chemicals and have funding to do something like that - they dont really need AI help here. There are plenty dangerous chemicals already well known to humanity and some dont actually take anything regulated to produce "except" complicated and expensive lab equipment.

Again difficulty of production of poisons and chemicals it's not what prevent mass murdering around the globe.

Complexity and cost are just two of the things that inhibit these attacks.

Three letter agencies knowing who's buying a suspicious quantity from the list of known precursors, that stops quite a lot of the others.

AI in general reduces cost and complexity, that's kind of the point of having it. (For example, a chemistry degree is expensive in both time and money). Right now using an LLM[0] to decide what to get and how to use it is almost certainly more dangerous for the user than anyone else — but this is a moving goal, and the question there has to be "how to we delay this capability for as long as possible, and at the same time how do we prepare to defend against the capability when it does arrive?"

[0] I really hope that includes even GPT-4 before the red-teaming efforts to make it not give detailed instructions for how to cause harm

There's been a fair amount of research into hooking up LLMs with the ability to call APIs, browse the web, and even control robots, no? The barrier between planning and doing is not a hard one.

As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.

A key question is whether the control problem gets more difficult as the model gets stronger. GPT-4 appears to be self-aware and passing the mirror test: https://nitter.net/AISafetyMemes/status/1729206394547581168#...

I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.

I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)

This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.

The image is OCR'ed and that data is fed back into the context. This is no more interesting or indicative of it passing the mirror test than if you had copy and pasted the previous conversation and asked it what the deal was.

I mean, you're just describing how it passes the test. That doesn't make it less impressive. Passing the test is evidence of self-awareness.

I can think of several ways that AI assistance might radically alters both attack and bodyguard methods. I say "might" because I don't want to move in circles that can give evidenced results for novel approaches in this. And I'm not going to list them for the same reasons I don't want an AI to be capable of listing them: while most of the ideas are probably just Hollywood plot lines, there's a chance some of them might actually work.

> Reducing the cost of assassination to ~$0 is quite another.

It is reducing the cost of developing an assassination plan from ~$0 to ~$0. The cost of actually executing the plan itself is not affected.

See here: https://news.ycombinator.com/item?id=38470768

Good planning necessarily reduces the cost of something relative to the unplanned or poorly planned version. If it identifies a non-obvious means of assassination that is surprisingly easy, then it has done something "close enough" to reducing the cost to $0.

This is where I have issues with OpenAIs stated mission

I want AI to be amoral, or rather I should say I do not want the board of OpenAI, or even the employee of OpenAI choosing what "moral" is and what is "immoral" especially given that OpenAI may be superficially "diverse" in race, gender, etc, but they sure as hell are not politically diverse, and sure has hell do not share a moral philosophy that is aligned with the vast majority of the population of humanity given the vast majority of humanity is religious in someway and I would guess the majority of OpenAI is at best agnostic if not atheist

I do not want a AI Wikipedia.... aka politically biased to only 1 worldview and only useful for basic fact regurgitation like what is the speed of light

Medical transcriptionists out of jobs? As far as I'm aware, medical transcription is still very much the domain of human experts, since getting doctors to cater their audio notes to the whims of software turned out to be impossible (at least in my corner of the EU).

My mom had to do this for her job and apparently some of the docs are so mumbly they have to infer a lot of the words from context and type of procedure but there is a lot of crossover everywhere so it depends a lot on which doc is mumbeling what. And yes you need special training for it (no medical degree though)

FWIW I had a doctor's appointment just this year with a transcriptionist present. (USA)

Did they really create a ton more jobs? The past few rounds of industrialization and automation have coinved with plagues/the black death that massively reduced the population, mass agitation, increasing inequality, and recently a major opioid epidemic in regions devastated by the economic changes of globalization and industrialization. I think these tools are very good and we should develop, I also think it's delusional to think it'll just balance out magically and dangerous to expect our already failed systems to protect people left behind. Doesnt exactly look like they worked any of the previous times!

Would you rather:

1) be surprised by and unprepared for AGI and every step on the path to that goal, or

2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it?

There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why.

The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.

It seems that silicon valley is developing not a conscious sentient piece of software rather a conscience, a moral compass is beginning to emerge.

After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures.

AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).

Yeah, I totally agree! Like, this is such an obviously true and valid reason to fire him, but they never came out and said it! So ... is this not what it actually was? Or ... what? It truly is mystifying.

From a doomer/EA perspective, publicly saying that GPT5 is AGI or such would likely inspire & accelerate labs around the world to catch up. Thus it was more "Altruistic" / aligned with humanity's fate to stay mum and leave the situation cloudy.

No, it would not; they are already trying to catch up.

Having an existing example showing that something difficult is possible causes everyone else to replicate it much faster, like how a bunch of people started running sub-4-minute miles after the first guy did it.

If you know the outcome is favourable then you go all in. Right now the other competitors are just trying to match GPT4, if they knew AGI was achievable then they would throw everything they have at it in order to not be left out.

Show me the researchers and companies where the people in charge and the people doing the work don’t think it’s possible to get better than GPT-4 and are slow-rolling things.

I suppose maybe there are ones where they are slow-rolling because of their opinions about existential risk of AGI. But that’s not contingent on what OpenAI says or does.

And I can show you companies that have multiple other costly research projects. All in means all in.

I think they didn't anticipate sucha a large backlash, especially from investors. They felt the backlash threatens OpenAI, both its non-profit and for-profit arms, so they reverted their decision, which in my opinion was a mistake, but time will tell.

Yeah, I guess so. I keep waffling between thinking it's some 4d chess thing, and thinking it's just normal human fallibility, where the board just made a massive mistake in predicting how it would go. But I just struggle so much to imagine that, because everyone I know in the industry, regardless of their level of expertise or distance from OpenAI, immediately knew how big a deal it was going to be, when we heard he was hired. But supposedly the people on the board had no idea? I think this might be the right conclusion, but I nevertheless struggle to fathom it.

The board had the right non-profit long-term focus, but it didn't have the willpower to communicate and realize their vision. Ilya is a wonderful scientist, but not a great leader.

Yep, you nailed it, I think. Pity.

so much this, he kept introducing clauses in contracts that tied investments to him, and not necessarily to openai. He more or less did it with microsoft, to a small degree. So his firing could have caused quite a lot of money to be lost. But ok no big deal.

But then he tried to do it again with a saudi contract. OpenAI board said explicitly they didn't want the partnership, and especially not tied to Altman personally being the CEO as a clause.

Altman did it behind their back -> fired.

This is the rumour on the streets, unconfirmed though

If they gave a reasonable reason to the public, they could get away with it. Shady CEO vs equally shady board.

My take is that the board probably never had a chance no matter what they said or did. The company already "belonged" to Altman and Microsoft. The board was just there for virtue signaling and for quite a while already had no real power anymore beyond a small threat of creating bad publicity.

I think they could have actually blown up the whole thing and remained in charge of a greatly-diminished but also re-aligned non-profit organization. A lot of people (like me) would have thought, holy crap, that was insanely bold and unprecedented, I can't believe they actually did that, but it was admirably principled.

Instead it was a confusing shambles that just left them looking like idiots with no plan.

How is public relevant here?

For one thing, we vote for the governments who are now definitely going to heavily regulate them after this fiasco.

But more generally, public perception just is important to any large and broadly targeted enterprise, because "the public" approaches "our customer base" as such an enterprise scales up. Think of companies like Google or Microsoft or Meta or Amazon; essentially everyone (in the US) uses their services in some form or another, so the perception of "the public" is indistinguishable from the perception of "the people who use our services". ChatGPT isn't quite at that point yet, but it's close enough.

Regardless of the board's failure to make their case, recent news suggests that the SEC is going to investigate whether it is true that Altman acted in the manner you describe, which would be a violation of fiduciary duty.

I agree that it seems like an open & shut case.

Typical SEC timelines mean that this will go public in about 18 months from now.

    An anonymous person has already filed an SEC whistleblower complaint about the behavioral pattern of Altman and Nadella, which has SEC Submission Number 17006-030-065-098.

https://pressat.co.uk/releases/ai-community-calls-for-invest...

    As the quid pro quo favoritism allegations remain under investigation, it is crucial to note that they are as yet unproven, and both Altman and Nadella are presumed innocent until proven guilty.

https://influencermagazine.uk/2023/11/allegations-of-quid-pr...

11 hours ago, the SEC tweeted the following new rule, which could be interpreted as a declaration that if Altman and Nadella are found guilty in this case, the SEC will block certain asset sales by OpenAI until the conflict of interest is unwound / neutralized:

    The Commission has adopted a new rule intended to prevent the sale of asset-backed securities (ABS) that are tainted by material conflicts of interest.

    Washington D.C., Nov. 27, 2023 — The Securities and Exchange Commission today adopted Securities Act Rule 192 to implement Section 27B of the Securities Act of 1933, a provision added by Section 621 of the Dodd-Frank Act. The rule is intended to prevent the sale of asset-backed securities (ABS) that are tainted by material conflicts of interest. It prohibits a securitization participant, for a specified period of time, from engaging, directly or indirectly, in any transaction that would involve or result in any material conflict of interest between the securitization participant and an investor in the relevant ABS. Under new Rule 192, such transactions would be “conflicted transactions.”

https://twitter.com/SECGov/status/1729895926297247815

https://www.sec.gov/news/press-release/2023-240

More information:

    The Company exists to advance OpenAI, Inc.’s mission of ensuring that safe artificial general intelligence is developed and benefits all of humanity. The Company’s duty to this mission and the principles advanced in the OpenAI, Inc. Charter take precedence over any obligation to generate a profit.

https://stratechery.com/2023/openais-misalignment-and-micros...

    Some analysts, including Stratechery writer Ben Thompson, have described the 2019 acceptance of Microsoft’s controversial investment by Altman as the beginning of a troubling pattern of Altman repeatedly making deals with Microsoft which were often unfavorable to OpenAI. ... As Thompson describes it, this pattern of behavior culminated in an unusual intellectual property licensing arrangement which Microsoft’s Investor Relations site describes as a “broad perpetual license to all the OpenAI IP developed through the term of this partnership” “up until AGI” (Artificial General Intelligence). This perpetual license agreement includes the technology for OpenAI’s flagship products GPT-4 and Dall•E 3. https://www.microsoft.com/en-us/Investor/events/FY-2023/AI-Discussion-with-Amy-Hood-EVP-and-CFO-and-Kevin-Scott-EVP-of-AI-and-CTO

https://michigan-post.com/redditors-from-r-wallstreetbets-ca...

    U.S. Securities and Exchange Commission – Tips, Complaints, and Referrals – Summary Page - Submitted Externally – PDF excerpt obtained 2023-11-26 via Signal
    Submission Number (redacted) was submitted on Wednesday, November 22, 2023 at 12:18:27 AM EST
    This PDF was generated on Wednesday, November 22, 2023 at 12:28:38 AM EST
    Image above includes ... the heading of a 7-page PDF titled "TCRReport (1).pdf" which was received by this reporter over the weekend via Signal.

https://www.outlookindia.com/business-spotlight/sec-consider...

Very interesting, thanks for posting this.

Interestingly, it appears that Microsoft has been caught violating SEC regulations at least 6 previous times, and that's just what I turned up in a cursory search:

2023: https://www.theregister.com/2023/02/03/activision_sec_settle...

2019: https://techcentral.co.za/eoh-microsoft-ensnared-in-sec-corr...

2013-2019: https://fcpa.stanford.edu/investigation.html?id=156

2013-2016: https://www.seattletimes.com/business/microsoft/ex-microsoft...

1999: https://www.itprotoday.com/windows-8/sec-investigating-micro...

2002: https://www.computerworld.com/article/2575713/microsoft-sec-...

And yet, they continue to exist and no CEO of MS ever stepped down because of any of these.

And I predict that even if Microsoft is going to be caught again that it will be a non-event in terms of actual repercussions. If Nadella exits MS HQ in Seattle in handcuffs I would be most surprised.

Yeah. I think the most realistically-achievable positive outcome is that Microsoft is forced to give up their new board-observer seat, which seems highly improper for them to have since the board they are observing is supposed to make decisions that benefit all of humanity equally. If Microsoft gets to have a fly on the wall in those discussions, it gives them a gold mine of juicy insider knowledge about which sectors of the economy are about to be affected next by Generative AI — knowledge which they can use to front-run the market by steering the roadmaps of Bing, MS Office, etc. so as to benefit from upcoming OpenAI product launches before any non-insiders are aware of what OpenAI is currently planning.

Microsoft plainly shouldn't be allowed to have this advantage, in that giving an advantage to any one party directly harms the mandate set forth in OpenAI's Charter:

    Broadly distributed benefits

    We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.

    Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.

https://openai.com/charter

It certainly seems that Microsoft, a "stakeholder", has managed to get a highly improper listening seat that will give them the ability to act on insider information about what's coming next in AI, allowing Microsoft to front-run the rest of the AI software industry and all those industries it affects, in a way that will plainly "compromise broad benefit". (Since any wealth that accrues excessively to Microsoft shareholders is not distributed to other humans who don't hold Microsoft shares.)

A mere 10 days ago, Nadella was shamelessly throwing his weight around on national TV, by appearing on CNBC where he improperly pressured the OpenAI non-profit board — which owes nothing to him legally or morally — to give him more deference, in direct violation of the "always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit" provision of the OpenAI non-profit charter.

https://www.cnbc.com/2023/11/20/microsoft-ceo-nadella-says-o...

I doubt Microsoft will be punished for their role in this (it's not clear to me that they did anything wrong...), but I'll be surprised if OpenAI is allowed to still exist as a non-profit a couple years from now.

Of course they'll still be allowed to still be a non-profit. If they weren't "allowed" to, that would be a total victory for the Altman-aligned faction that sought to corrupt OpenAI from within and cause it to abandon its charter, and a total loss for Elon Musk and others that donated a total of $133M to the non-profit due to its charter. https://techcrunch.com/2023/05/17/elon-musk-used-to-say-he-p...

Musk and other donors are the most obvious aggrieved parties, and it is also arguably the case that every member of humanity has standing to sue for violation of the charter, because the charter explicitly declares that the primary fiduciary duty of OpenAI, Inc. (which is a non-profit) is to humanity broadly. (Therefore, every human is financially harmed by any charter violation, with such harm manifesting as a reduction in the net present value of the future benefits each human will receive from safe, broadly beneficial AGI.)

The SEC's role in fiduciary misconduct cases is not to rewrite a company's charter - that would be extremely improper and the opposite of their mandate. The SEC's mandate is to be the protectors of the status quo and of the original intent of the organizers of an entity. In this case, that means they will seek to protect the OpenAI non-profit's charter from efforts by Sam and others to erode the charter's power in violation of Sam's fiduciary duty to said charter.

The SEC's job in fiduciary misconduct situations is to remedy the situation by reversing improper governance decisions and forcing the fiduciary (Sam) to uphold their legal duty, which in Sam's case is his contractually bound duty to uphold the Charter of the non-profit entity named OpenAI, Inc. https://en.wikipedia.org/wiki/OpenAI#:~:text=the%20non%2Dpro....

OpenAI states this very clearly in multiple places on their website. For example:

"each director must perform their fiduciary duties in furtherance of its mission—safe AGI that is broadly beneficial" - https://openai.com/our-structure

If you would like to read about the types of remedies available in fiduciary duty violation cases, I recommend this resource:

Book Chapter:

    REMEDIES FOR BREACH OF FIDUCIARY DUTY CLAIMS

https://m.winstead.com/portalresource/lookup/poid/Z1tOl9NPlu...

For example (quoting from the book chapter above):

    C. Permanent Injunction
    A breach-of-fiduciary-duty plaintiff may be entitled to an award of a permanent injunction as a remedy. ...
    The purpose of an injunction is to remove the advantage created by the wrongful act.

In the context of this suit, the permanent injunction or injunctions could block all of the following:

    • Permanently prohibit Microsoft from holding any board seat or board observer seat on the OpenAI board
    • Permanently prohibit Microsoft from making mass employment offers to OpenAI staff (a practice which, in the case of bad faith situations such as this one, is known as "workforce raiding")
    • Permanently prohibit Sam Altman from owning Microsoft stock or derivatives thereof
    • Permanently prohibit Microsoft from receiving early indications of OpenAI research & product roadmaps earlier than the general public
    • Permanently prohibit Sam Altman from receiving job offers at Microsoft or any of its subsidiaries until after a sufficient cooling-off period has elapsed since departing OpenAI, to limit future occurrences of the "revolving door" mechanic documented here:

https://news.ycombinator.com/item?id=38387518

I dunno, seems more likely that they'd shutter a nonprofit that's a clear sham and assess a bunch of taxes and penalties on the for-profit subsidiary that was the beneficiary of the sham. But beats me!

It was supposed to have two functions: tax dodge + pull the wool over the eyes of regulators. It may fail on both counts now that they've shown that the nonprofit wasn't functional in its stated capacity. Win the battle, possibly lose the war?

Yeah this is what I think as well. Certainly regulation won't be avoided now, and regulators will (or should) know not to trust Altman to contribute to developing it.

Yep, no doubt there are more shoes left to drop.

The Michigan Post article is mostly speculation and that publication doesn’t have much depth/history to it. Check out their “advertise with us” page.

This whole info dump feels like a mishmash of links to thoughtful things (Stratechery) with links to speculative articles that are clearly biased. Like how is “Influencer Magazine” breaking a story that Wall Street Journal and Kara Swisher are overlooking?

I don’t mean to be a jerk. Just really unconvinced.

I would guess that "WLW FUTURE PRESS RELEASE DISTRIBUTION" is a publicist service that was hired by the person making the whistleblower complaint. upwardbound claims there's a lot of money in being a successful whistleblower: https://news.ycombinator.com/item?id=38388246

I don't know if I find the Microsoft/OpenAI favor trading allegation that persuasive (unless new information is uncovered, e.g. Microsoft letting Sam use a private jet or something like that). However if the SEC actually ends up enforcing the "fiduciary duty is to humanity" thing in OpenAI's charter at some point, that would be incredibly sweet.

| if the SEC actually ends up enforcing the "fiduciary duty is to humanity" thing in OpenAI's charter at some point, that would be incredibly sweet.

Absolutely. It's 100% their job.

"Asset-backed securities" doesn't sound like corporate equity to me.

In a broad reading it could easily be just that.

Securities class contains stock

Stock = backed by the company balance sheet

What would be an example of a non-asset-backed security?

In the financial world asset backed securities are typically anything that isn't a mortgage, I think that some crypto would qualify as well as the rights to future profits on some venture.

Argue their case? To whom? They were the board.

To the stakeholders, which include employees, customers, partners and, by OpenAI own mission statement, all of humanity in general.

To the public, and to employees.

To what end? The public and the employees don’t have a say in the corporate governance. That is the function of the board.

As far as I can tell, the board had no obligation to consult the public or their shareholders or their employees on any of this.

And how did that attitude work out for them? Upwards of 90% of their staff threatened to bail out, every single one of them look like fools in public, and I would be shocked if there were not some recriminations behind closed doors from the likes of Microsoft's CEO.

It’s not an “attitude.” It’s the legal structure of the corporation’s leadership. They were no more capable of incorporating the public will than an individual is capable of taking a vote on the flow of traffic on a public highway. That is to say, even if they had done what is proposed here, it wouldn’t have made a difference.

In general, your comments strike me as reflecting a very naive understanding of how anything works. Legal responsibilities are simply not the only thing that matters. This entire episode was about 95% a PR battle, which the board lost and Altman won. If it's not obvious to you that the board's legal rights and responsibilities had essentially zero to do with the outcome here, then I really don't know how to help you.

“Naive” is thinking that taking a poll of the public or employees (which is the only reasonable action I can think of that looks like “arguing their case”) would have had any positive effect on the outcome. This is simply… not how decisions are made. Even shareholder vote calls (for companies that have them) are coordinated, with the outcomes for consequential decisions well understood before voting happens. The optics for Open.ai certainly should have been better, but there is no version of this story where the board doesn’t do whatever they hell they want, and no version where “making the case” doesn’t result in even more chaos.

Corporations are not a democracy. They do not “owe” the public any information that they aren’t compelled to disclose. And they certainly don’t “argue their case” to some nebulous forum comprised of either the public or employees.

When has that ever happened?

>there is no version of this story where the board doesn’t do whatever they hell they want, and no version where “making the case” doesn’t result in even more chaos.

The failure to make their case is PRECISELY why the board was ultimately unable to do what the hell they wanted: remove Sam Altman. As it turned out, his presence was important enough to the employees that the usual corporate playbook of "do whatever the hell we want and only disclose what is legally required" backfired spectacularly.

If your thesis was correct, Sam would not be there right now.

From reading your comment I think this might actually be a simple language misunderstanding.

You say:

> taking a poll of the public or employees

And then:

> which is the only reasonable action I can think of that looks like “arguing their case”

But that is a total non sequitur. "Taking a poll" has no relation whatsoever to "arguing their case". So I think you might not know what "arguing their case" means.

So I'll just plainly say what I think they should have done, without any jargon. I think they should have, after firing him, released a statement targeted at employees but also with a public audience in mind, something like this:

"We have simply lost faith in Mr. Altman to faithfully execute his duties as the executive of OpenAI's non-profit charter. We believe he has been acting in the interests of his own, which are not aligned with our mission. We have tried to redirect his efforts over a period of time, without success, and now are taking the only recourse that we believe is available to us to fulfill our duty to the organization. We understand that many of you will find this jarring and unsettling, but we hope you will continue to believe in the mission of OpenAI and stick with us through this trying and uncertain time, so that we can come out of it stronger and better aligned than ever."

Then anyone who quit would at least need to rationalize - to themselves, and to their social circles - why they chose not to take that to hear. Maybe for many / most / all of them, just "money" or a personal loyalty to Altman would have still won the day, but it certainly wouldn't have been as easy as it was to abandon a board that was seen as confusing and shambolic and refusing to explain itself.

That statement above is the part that would be "arguing their case". Just making the statement; the statement is the argument for that they did. Note that it doesn't include any sort of polling of anyone, or any different use of their legal rights or responsibilities. It's "just" PR, but that actually matters a lot.

And just to respond more narrowly to:

> [Corporations] certainly don’t “argue their case” to some nebulous forum comprised of either the public or employees.

> When has that ever happened?

It happens all day every day. This is what PR is. Surely you're aware of the existence of PR and that it is not infrequently utilized?

To the public: For the normal PR reasons that every large enterprise is subject to caring about. To employees: I suspect substantially less than the like 95% of employees who came down against the board would have done so if the board had explained what they were thinking. There are probably at least a few people there who are not comfortable viewing themselves as rank mercenaries in service to a profit-and-power-motivated game-player, and actually joined OpenAI with some belief in its mission. But maybe not! I dunno.

Really hope details come about this with all perspectives being provided.

Whether they stuffed up or there are some details that made the situation unworkable for the board, it’s an interesting case study in governance and the whole nonprofit with a for profit subsidiary thing.

For me it seems to be a debate between moral versus money. Is it morally correct to create technology that would be extremely profitable but has the potential to fundamentally change humankind.

Makes me curious if the reason behind that is just an NDA.

Reply on news.ycombinator.com