Comment by hn_throwaway_99 | Hacker News Reader

Comment by hn_throwaway_99 | Hacker News Reader

Parent Full thread

hn_throwaway_99

I mean, here is a relevant passage from the paper, linked in another comment: https://news.ycombinator.com/item?id=38373684

If I were the CEO of OpenAI, I'd be pretty pissed if a member of my own board was shitting on the organization she was a member of while puffing up a competitor. But the tone of that paper makes me think that the schism must go back much earlier (other reporting said things really started to split a year ago when ChatGPT was first released), and it sounds to me like Toner was needling because she was pissed with the direction OpenAI was headed.

I'm thinking of a good previous comment I read when the whole Timnit Gebru situation at Google blew up and the Ethical AI team at Google was disbanded. The basic argument was on some of the inherent incompatibilities between an "academic ombudsman" mindset, and a "corporate growth" mindset. I'm not saying which one was "right" in this situation given OpenAI's frankenstein org structure, but just that this kind of conflict was probably inevitable.

Just spot checking: did anyone comment on this paper when it was published? Did any media outlet say “hey, a member of the OpenAI board is criticizing OpenAI and showing a conflict of interest?” Did any of the people who cover AI (Zvi, say) discuss this as a problem?

These are serious questions, not gotchas. I don’t know the answers, and I think having those answers would make it easier to evaluate whether or not the paper was a significant conflict of interest. The opinions we have formed now are shaped by our biases about current events.

It didn’t make HN.

Gambling that no one reads academics papers by fringe Open AI board members no one has heard of before is probably a safe bet but it's still a risk if some doomer AI people on Twitter pumped it up, some journo discovered the tweets and sells it as "concern in the industry including one of OpenAI's own board" and it got swept up in some lawyer-style grilling by Congress which Sam just had to go through.

Who cares if the paper was covered in the media or not? Think tanks write policy papers for regulators, not HN. And it's the regulators that Sam was worried about. From the article:

> Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, particularly at a time, he added, when the Federal Trade Commission was investigating OpenAI over the data used to build its technology.

> If I were the CEO of OpenAI, I'd be pretty pissed if a member of my own board was shitting on the organization she was a member of while puffing up a competitor.

Considering what's in the charter, it seems like she didn't do anything wrong?

> We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.

>We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.

more here: https://news.ycombinator.com/item?id=38372769

I agree and none of the other passages people are quoting from the paper (which I admittedly haven't read yet) seem at all controversial. She's saying OpenAI's messaging about AI safety is not landing because it's simultaneously launching products that are taking the spotlight, and Anthropic is doing a better job at signaling their commitment to safety. That's true, obvious, and entirely in line with the charter she's supposed to uphold.

> Considering what's in the charter, it seems like she didn't do anything wrong?

It’s incredibly disingenuous to slap your name on an ethics paper claiming a company is doing malfeasance such as triggering a race-to-the-bottom for AI ethics when you have an active hand in steering the company.

It’s shameful. Either she should have resigned from the company, for ethical reasons, or recused herself from the paper, for conflict of interest.

> when you serve on the board of directors for that company

I'm not sure if this is the true statement, because the not-for-profit/profit arms make it more complex.

In this case, the non-for-profit board seems to act as a kind of governance over the profit arm, in a way, it's there to be a roadblock to the profit arm.

Normally a board aligns with the incentives: to maximize profit for shareholders.

Here the board has the opposite incentive, to maximize AGI safety and achievement, even at the detriment of profit and investors.

She is at the top of the pyramid. Did they not fire the chief executive? I am saying she is morally culpable for OpenAI’s actions as a controlling party.

To put these claims in a published paper in such a naive way with no disclosure is academically disingenuous.

She's not in charge of the for-profit arm though, all she could do was fire the CEO, and she did, which would seem consistent with her criticism. I don't think she has many more power as being on the board. She also isn't at the top; in the sense she needs other board members to vote similar to her to enact any change, so it's possible she kept bringing up concerns and not getting support.

Academically, did she not disclose being on the board on her paper?

Her position was listed as a fun fact, not in a responsible disclosure of possible conflicts of interest (though it ran the other way).

Being at the top of the org and being present during the specific incidents that gives one qualms burdens one with moral responsibility, even if they were the one who voted against.

You shouldn’t say “they did [x]” instead of “we did [x]” when x is bad and you were part of the team.

It sounds like your argument is "Even if OpenAI did something bad, Helen should never write about it, because she is part of OpenAI".

Or, that she should write her paper in the first person: "We, OpenAI, are doing bad things." That would probably be seen as vastly more damaging to OpenAI, and also ridiculous since she doesn't have the right to represent OpenAI as "we".

I have no idea why you think that should be a rule, aside from wanting Helen to never be able to criticize OpenAI publicly. I think it's good for the public if a board member will report what they see as potentially harmful internal problems.

I just don’t know why an ethicist would remain involved in a company they find is behaving unethically and proceed with business as usual. I suppose the answer is the news from Friday, though the course feels quite unwise for the multitude of reasons others have already outlined.

Regarding specific verbiage and grammar, I’m sure an academic could give clearer guidance on what is better form in professional writing. What was presented was clearly lacking.

One thing we've learned over the past few days is that Toner had remarkably little control over OpenAI's actions. If a non-profit's board can't fire the CEO, they have no way to influence the organization.

You’ll catch more flies with honey than vinegar.

Did we read different things? All it said was that they had been accused of these things, which is true. If your charter involves ethical AI I’d imagine the first step is telling the truth?

From the PDF:

While the system card itself has been well received among researchers interested in understanding GPT-4’s risk profile, it appears to have been less successful as a broader signal of OpenAI’s commitment to safety. The reason for this unintended outcome is that the company took other actions that overshadowed the import of the system card: most notably, the blockbuster release of ChatGPT four months earlier. Intended as a relatively inconspicuous “research preview,” the original ChatGPT was built using a less advanced LLM called GPT-3.5, which was already in widespread use by other OpenAI customers. GPT-3.5’s prior circulation is presumably why OpenAI did not feel the need to perform or publish such detailed safety testing in this instance. Nonetheless, one major effect of ChatGPT’s release was to spark a sense of urgency inside major tech companies.149 To avoid falling behind OpenAI amid the wave of customer enthusiasm about chatbots, competitors sought to accelerate or circumvent internal safety and ethics review processes, with Google creating a fast-track “green lane” to allow products to be released more quickly.150 This result seems strikingly similar to the race- to-the-bottom dynamics that OpenAI and others have stated that they wish to avoid. OpenAI has also drawn criticism for many other safety and ethics issues related to the launches of ChatGPT and GPT-4, including regarding copyright issues, labor conditions for data annotators, and the susceptibility of their products to “jailbreaks” that allow users to bypass safety controls.151 This muddled overall picture provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent.

> This […] provides an example of how the messages sent by deliberate signals can be overshadowed by actions that were not designed to reveal intent.

What a lovely turn of phrase! I'm stealing it for use later, in place of "actions speak louder than words".

Well since that's not what the paper claims at all...

The paper itself claims that OpenAI's actions have undone their stated goals:

https://news.ycombinator.com/item?id=38374972

It has an excess amount of weasel words, so you might need to employ ChatGPT to read between the lines.

Reply on news.ycombinator.com