Hacker News Reader

Hacker News

Comment by INGELRII | Hacker News Reader

Parent Full thread

INGELRII

Seven Signs of Ethical Collapse

1. Pressure to maintain numbers

2. Fear and silence

3. Young ‘uns and a bigger-than-life CEO

4. Weak board of directors

5. Conflicts of interest overlooked or unaddressed

6. Innovation like no other company

7. Goodness in some areas atones for evil in others

from "The Seven Signs of Ethical Collapse: How to Spot Moral Meltdowns in Companies Before It's Too Late", 2005 by Marianne M. Jennings, professor of legal and ethical studies in business in the Department of Management in the W.P. Carey School of Business at Arizona

View on HN

Letter signing looks like a masterful use of the bandwagon effect. Quickly reframing the issue, going to attack, and creating a sense of outrage and hurry. People start quickly pressuring each other because there is a need to be unity for the scheme to work and no time to think.

> Within hours, messages dismissed the board as illegitimate and decried Altman’s firing as a coup by OpenAI co-founder and chief scientist Ilya Sutskever, according to the people.

>people identified as current OpenAI employees also described facing intense peer pressure to sign the mass-resignation letter.

>“Half the company had signed between the hours of 2 and 3am,”

This is also consistent with some number of OpenAI operators wanting Altman removed, and then discovering that the company's long-term financial outlook, and their accrued compensation in particular, was highly contingent on him staying. It's pretty normal for people to believe that they could, if all else was equal, have a better or more healthy management culture. But people rarely act on that belief precisely because earnestly pursuing it can end up threatening everyone's future well-being.

It's also consistent with OpenAI's board comprehensively botching their response to those concerns, even if had been clear that over the medium term Altman needed to be managed out. That's why there's a term for this ("managed out").

Instead, the OpenAI board created news cycle after news cycle of palace intrigue by seeming obliviousness to how their actions would be read.

Incentives are the force behind almost everything.

"Well, I think I’ve been in the top 5% of my age cohort all my life in understanding the power of incentives, and all my life I’ve underestimated it. And never a year passes but I get some surprise that pushes my limit a little farther." -- Charlie Munger

The board fired Altman and maintained discretion. It was Altman that instigated the circus after that.

And?

How do you distinguish between bandwagoning and genuine consensus? How unpopular does someone need to be before they can cry "bandwagoning" whenever they're unaminously criticized?

Speed. Consensus usually implies self-reflection. Know thyself and all that jazz. If there's no time to know thyself, it's hard to get true consensus.

I really dislike the current attitudes around ethics and AI. Tools should be 100% open about what they can and can't do. Building unethical software like deepfakes for porn is also unethical.

Quoting of vague "ethics" risks around AI, or neutering LLMs in the vein of making them "safer" seems dubious to me. Obviously, there are a huge number of problems which depend on safe AI - where safe often means something completely different from what is currently being talked about. e.g. does the AI reliably do what I expect it to do in the face of adversarial inputs? does it handle risky settings like deciding to refund customers? How can I ensure that the AI is making legal decisions and not secretly using redlining, or other illegal criteria to make its decision?

I am still taking in information and trying to figure out my stance for AI ethics, but this is moreso about the ethics of the business and its structure, not the ethics of the AI that they make.

Both of these are intertwined, if not outright the same thing. A place that has no corporate understanding of ethics, no staff to take care about that, or even fires that staff (like Twitter did [1]), cannot build any ethical system. No matter if it's chatbots that shouldn't scoop up abuse by 4chan [2], reproduce long-debunked racially charged myths [3], or even something as simple and innocent-seeming as a soap dispenser that had never even been tested by Black people [4].

Many people think like this is "wokeness" and at best a waste of money, and yet time and time again there are incidents that show in sometimes very horrible ways just how bad biases and a lack of diversity and oversight at all stages - planning, development and post-launch support - can end.

IMHO, ethics oversight and threat modelling based on ethics should be seen, similar to pentesting and QA in AI, software (and, as the example shows, also hardware) development, as an absolute requirement to do - because if you don't do it, 4chan will, and tear your public image apart in the process.

[1] https://www.cnbc.com/2023/05/26/tech-companies-are-laying-of...

[2] https://www.theverge.com/2016/3/24/11297050/tay-microsoft-ch...

[3] https://fortune.com/well/2023/10/20/chatgpt-google-bard-ai-c...

[4] https://www.standard.co.uk/news/world/automatic-soap-dispens...

I agree that oversight is needed - but I'm not certain people are the right approach to this oversight. Everything described above represents an undesirable software output for the given business. Why can't we make the LLM smart enough to know it's current operating context and build guardrails to catch the cases which slip through?

The proper guardrails worthy of being entrusted, they best come from an ecology of human minds with diverse perspectives and ideas, to grok the whole possibility space with minimal blind spots.

The human machine is already tuned to do this navigating of the information landscape. The sense of curiosity itself lures us toward things that are just a little weird -- the goldilocksian zone between familiar and unfamiliar[1]. Our minds are designed to bridge gaps and seek paths into otherness, but only when the individual steps are small. We are built to seek, grow understanding and incorporate diverse thought :)

[1]: https://nautil.us/curiosity-depends-on-what-you-already-know...

Thing is, you need people to take care in selecting training data. GIGO - garbage in, garbage out - especially applies in AI.

Reply on news.ycombinator.com