Hacker News Reader

Comment by hn_throwaway_99 | Hacker News Reader

These sorts of adjunct, off-to-the-side "ethics" teams are always corporate window dressing, and when push comes to shove they always get jettisoned when competition demands an increase in speed.

I don't mean to start a flame war, but this is a reason why I think "Chief Diversity Officers" are such an ill-conceived notion. It's not that I don't think diversity in corporate environments is extremely valuable, but it's that a Chief Diversity Officer doesn't really "own" any sub-organization of the business. That is, the Chief Product Officer owns the product teams, the Chief Technical Officer owns the engineering teams, the Chief Marketing Officer owns the marketing teams. Even the Chief Human Resources Officer, while largely providing support to other teams, still has sizable staff of their own to get their job down. A Chief Diversity Officer's job largely comes down to telling other teams how they should change or structure their teams. I have yet to see that approach work.

Different teams can certainly work to a common corporate goal, even if that goal isn't the primary focus of a particular team. If you really want to ensure that diversity is valued, any initiatives should actually be led by the teams who also do the main "work of the business", and the goals should be championed by the executives of these teams.

So, in the vein of this article, I think independent "ethical AI teams" are a really bad idea. Better to define what ethical AI means at a high leadership level, and if teams need particular experts in these areas they should just hire them as part of that team, not off as a part of some dysfunctional "ombudsman" role.

View on HN

This.

People should understand AI safety != AI ethics. AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion.

The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.

The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.

It seems increasingly that the concerns from the ethics branch have not come to pass at all. Stable diffusion does not default to creating black people (you have to insert black into the prompt), this has not caused mass social harm despite the tens of millions of users. Hence companies are unwilling to pay for these teams.

The other concerns, such as mass unemployment, or copyright of training material, are real, but they are better understood by economists, lawyers etc, rather than this awkward branch of AI ethics.

> this has not caused mass social harm despite the tens of millions of users

Isn’t it a bit early to tell? I can foresee ChatGPT being used to farm “karma” on social media sites to bot accounts to credibility on sites like Reddit - maybe even HN.

Yeah, this is like claiming lead ethyl in gas “hasn’t caused mass harm” in 1954.

There’s absolutely zero way to know that yet.

Except that lead's toxicity has been known for thousands of years.

Yes and the dangers of AI overlords have been known at least since the first Terminator movie /s.

Seriously though, people tend to dismiss things they are know are bad if there is no immediate harm. Certainly they had no idea how incredibly pervasive the lead from gasoline would be, that it would be detectable at harmful levels in basically everyone. It is also hard to measure these harms. How much crime of the 70s and 80s could be attributable for example.

Oh we instantly did know how bad it could get. General Motors,Standard Oil and DuPont knew for sure, they had industrial incidents and lied about them! US Surgeon General was bought, people with opposing opinions were sued.

It was instantly obvious to anyone that using leaded gas esp. in agriculture would be deadly and toxic. The risk was ignored for profit, PR handled, underplayed. Even the chronic effects were known!

We even had an early alternative of just adding more ethanol to the pettol. But no...

It took 50 years of scientists trying to convince public how deadly this thing is, with varied success. And 30 more years for the bulk of the phaseout.

> Oh we instantly did know how bad it could get.

Who are “we”?

The same could be told about AI - content farms, generating spam that bypass spam filter, etc…

Some people knew them, majority of the public don’t

The Public, which is not an entity, also knew enough about lead. But they were not told the gas contained lead for the longest time. Ethyl was marketed as an antiknock additive explicitly dropping lead from the name for this very reason.

Then there was a big chunk of time taken out for WW2 where such considerations were pushed out. The issue returned later but then it was argued there was no alternative for years.

Thanks, longer piece on that: https://www.bbc.com/news/business-40593353

Lead toxicity was limited due to it not being sprayed about with fumes out of the exhaust of every vehicle, and then number of those vehicles being low. It got quickly somewhat regulated away. (But not 100%.)

AI toxicity... We don't even know the risks. There's potentially no limit to the damage. We have some examples (metrics making echo chambers, automated classifiers giving people wolf ticket for no reason, amplified evil biases in conversational AI, fake news and deepfake generation) and some guesses for now. We do not even know how to begin to regulate it.

It's not even the same ballpark. Internet would be a closer comparison and it's still wild west.

Same risk as hiring a troll farm now

Troll farm is limited number of people with limited capability. These can be tracked down and shut down, their enployers as well. The effects of a troll farm are relatively predictable which is why they're used. It's somewhat rare that anything really unpredictable results from small scale social engineering of this kind.

I must strongly disagree here about unpredictable results from social engineering. Vladmir Lenin was deliberately let through Germany because they figured he would just weaken the tsar. It is hard to get smaller than one person or more unpredictable in impact with many others.

I would argue social media has at least 10 years of known toxicity, it is so addictive and harmful to young people i dont get why its not prohibited like gambling or smoking..

The problem with measuring the "harm" of something like AI vs e.g. lead is that with lead there are direct first order chemical harmful effects that are indisputable. With AI, any harm will get expressed via complex social dynamics, which involve so many other variables as well.

Which can be written off as laziness or just-world fallaciousness! Even consequences like "first order chemical harmful effects" are mostly ignored by the great-great-grandparent's "economists, lawyers etc"; the way AI will help concentrate money and power in the already overstuffed upper classes will barely be acknowledged.

Okay, so for leaded petrol substitute mortgage backed securities. The person who invented those was not going “buahahah, fly my pretties, cause a global financial crisis”; they probably thought they were doing a _good, ultimately of benefit to society thing_. And yet…

And therefore we also fail as a society to effectively regulate various complex financial instruments before they actually cause an obvious problem.

My point is not that we should or should not regulate AI. It's just that I believe the causal link from certain types of AI usage to concrete harm for society is too complex for enough humans to band together and achieve consensus a priori on regulation.

I'm certainly not looking forward to this next election of GPT-3 powered social media Super PACs, that's for sure. But I wouldn't get rid of it either.

Do you consider "karma" farming "mass harm"?

In answer, how frequently does "append Reddit" appear as a top comment on a "Google search is ad-optimized garbage now" article?

Perhaps this will force people to search on subject-specific forums instead of subsections of the everything forum.

That doesn't seem different-in-kind. Spammed GPT content will eventually infect them too.

I believe that it's a case where the difference in scale becomes the difference in kind. Reddit is easy to target because there is one single Reddit: GPT spambots farm karma by reposting porn and cute animal photos and easily sail past even the most stringent karma requirements after letting the account age for a month or three. Niche forums would require the bot to develop a positive reputation to be accepted as the authoritative answer in each new forum that is targeted.

Would they? I don't spend a lot of time on niche forums these days, but the question boils down to "Would you not accrue positive karma over time from posting?"

I'm sure there are forums out there sufficiently expert that's not the case... but that doesn't feel like most forums (even niche).

Exhibit A: me, who doesn't post particularly interesting things on HN, but eventually accrues karma simply from time

I just took a reverse Turing test with GPTina and she concludes I'm probably human because I'm idiosyncratic in my speech and imperfect in my grammar.

When I slop a grammer error or two it's ok it shows I'm not a bot. I used to check my grammar carefully but not anymore. Authenticity and stuff.

Who’s checking the karma of posters on Reddit?

Some subs have karma requirements for posting or commenting.

Some subs only allow historians, some subs only allow women, some subs only allow students at a specific university.

I don’t think one’s access to different subreddits is something anyone should waste brain cycles on, many bigger fish to fry :)

What's the societal harm in karma-farming. It was all fake when it was purely humans, anyway.

The main issue I see is terminology. Ethics is a broad field. “AI Ethics” has come to mean mostly eliminating different kinds of bias from AI models. However, AI raises multitudes of ethical questions, and merely being unbiased is far from enough for an AI model (or a person) to be acting ethically.

Whenever I hear the word “ethics” I always ask myself “whose”. I hate how people talk about ethics as if it was some non-controversial topic where everyone can clearly see what the “right” answers are.

This world features a competition between radically opposed ethical systems. We don’t agree on whether ethics is objective (i.e. some people’s ethics are superior to those of others) or subjective (i.e. “superior” is just another way of saying “same as my own”), and even those who agree that it is objective, often can’t agree on who is objectively more correct, even though they both agree that in principle someone must be.

I much prefer those who talk about ethics while making clear which brand of ethics they are selling - whether their name is John Rawls or Robert Nozick or Peter Singer or Edward Feser - than people who pretend we all agree on what is ethical, and leave their actual ethics as something to be inferred rather than openly advertised. From what I’ve seen, “AI ethics” fails at this completely

Often when they say ethics they mean politics, their politics specifically.

For me, democracy is a system where persons can vote for their interests. Different persons have different interests and that's good. You must compromise with others so everybody gets partly satisfied.

Others see democracy as a way to establish truth, those are the ones talking about ethics. Even if they lose, they still believe they must win, because they're right and the others wrong.

Partly satisfied is not ethical enough. An ethical version would end up with equitable compromises...

(I probably do not have to spell out economic implications of that.)

The brand is obvious- it's the author's ethics. Why shoehorn into someone else's brand? Sam Bankman-Fried claimed "Effective Altruiat" ethics. What is the value in demanding someone lie about their vrand?

Someone like Peter Singer - he’s very clear about what his first principles are, and that his first principles are very different from those of many other people. I think some of the conclusions he draws from those principles are horrific - e.g. painless infanticide should be legal with parental consent. But, whatever you think of him, nobody could accuse him of being non-transparent about what his principles are.

By contrast, these “AI ethics” people aren’t clear about what their first principles are, or how those principles differ from those of other people. In that regard, as horrible as Singer’s conclusions may be, he’s vastly more honest and transparent and self-aware than the average “AI ethicist”

AI alignment people are the group who are capable of bringing AGI to life, but want to do it responsibly.

AI ethics people are the group who are not capable of bringing AGI to life, but want to make sure that those who can do it responsibly or not at all.

There is no evidence that any group is capable of bringing AGI to life.

no evidence? none? Google Lambda and Bing/Sydney are not at least incrementally closer to evidence?

That's not AGI. An AGI would be able to tell you they don't know things, ask for clarification as to why you think they're wrong, etc.

ChatGPT just spouts out a different wild guess when you tell it it's wrong, but doesn't learn from its mistake—not even within a single chat. Sydney just goes full psychopath.

That's not AGI. AGI would be able to write its own code, and improve it exponentially.

That's a big assumption. No one has built an AGI yet so we don't know whether an AGI would be capable of improving it's own code exponentially. Any exponential growth function quickly runs into hard physical limits.

Not necessarily, much like every human does not know pharmacology, genetics, robotics or other specialties that could be used for self improvement. Including social engineering and programming.

What you're describing is a particular kind of superintelligence, recursively improving kind. Between that and general intelligence is another gap.

Humanity as a whole can improve itself indefinitely, there is little doubt about that. For robots, there is no distinction between a single robot and robotuity, as software can be replicated cheaply and quickly.

We're generally intelligent but we can't consciously rewire our brains.

We sort of can, through meditation, habit, study, and chemical alteration - obviously within constraints, barring future technological enhancements.

We can, this process is called "learning".

Practice and study are conscious behaviors. Learning is not.

That's Super Intelligent AGI, not merely AGI.

None of the existing models of what we call 'general intelligence' are capable of improving themselves exponentially.

> Google Lambda and Bing/Sydney are not at least incrementally closer to evidence?

No, in the same way that boiling water in a kettle is not incrementally closer to inventing the fuel for a rocket ship.

You can, in hidsight do a linear connection of water boiling -> steam engine -> fuel engine -> aeronautics -> space travel

But the first time someone boiled water, if you said that got you closer to being on the moon I think you would be mad. There is a non trivial chance that all our current work in AI is functionally useless in terms of creating AGI, it might take a complete breakthrough in ML, tech not invented yet, or it might not be possible at all.

Except, in your example, boiling water did get us closer to the moon :)

Yep, and there was a much mre promissing science on checking Ether, and the magical qualities of gravity defying physics.

Who is to say that our current ML models are not Ether studies and utterly useless.

If we crack AGI in 10 years, maybe our current models are the fuel engine. But if they are not even close, we might be boiling water. Or perhaps be wrong all together, thats why I said your observation of us getting closer thanks to those proyects is not right, at least not yet.

We have lots of examples of generally intelligent systems, they're common and they evolve by themselves, so the idea that it might not be possible at all just seems ridiculous.

Large language models might not be capable of general intelligence by themselves, but a model network that includes a LLM as part of a generalized game playing model would certainly have the potential to get there.

I think it is fair to say that "not at all possible" is perhaps too strong. On the potential of certain approaches I am less certain we have or know what we need.

But we also have examples of things all around us we cannot make ourselves (e.g., a simple animal or lifeform from scratch).

Except that we've already created synthetic genomes: https://www.nytimes.com/2019/05/15/science/synthetic-genome-...

If we can define what "generally intelligent" means in concrete terms, we can absolutely create it. The problem is that we don't have a good understanding/shared idea of what we're trying to achieve. We can create models that can learn to play many different games using the same weights, solve IQ/logic tests and pass the Turing test consistently, but people are going to move the goalposts and say those are just complex mashups of autocomplete and flowcharts.

To be honest I don't think AI skeptics are ever going to believe until there's a Skynet trying to exterminate them.

I don't share the view that all that is holding us back from creating AGI is a definition. Btw., that synthetic genome is a long way from making even something like an little nematode.

I rather think AGI "idealists" (maybe there is a better word) will forever hold out (any) algorithmic advances as a sign of the immanent creation of AGI proper.

AGI happens slowly then suddenly all at once, to paraphrase a saying. There is no other way it can happen.

Why not? I'd be interested in hearing the reasons.

It would have to be reproducible: one second of a snippet of a video clip of something; the next, the thing happening. You rewind it and play again. Scrub through and try to determine a different branch. At one timestamp, separation, peace, unknowing. In the next, flames and smoke and a flashing blur.

But no--the state of the world in that second versus the next could not have been predicted. It just happened, just so, and it would take longer to calculate the determinism than to adjust to the new.

Singularity-related religious beliefs.

1) progress doesn't happen linearly, it's very coarse stepwise function (remember alphago?) 2) acceptance of an AGI actually being created is also a non-linear function - look at current definitely-not-AGI LLMs: some people think it's close, some people say that absolutely not, these are big matrices of numbers regurgitating words in plausible combinations and that's all - and these people won't change their minds easily one way or the other, thus declaring delivering of AGI will be very controversial even if we could agree on the definition.

Ok so what’s your point? We can just ignore it? Or what.

AGI worries we probably could ignore until AGI exists but there is now an industry around them which has its own momentum.

This seems dumb to me. It’s like if the Japanese had the clairvoyant ability to know that nuclear bombs were being built and used and the immediate future and said “We can ignore it until it exists)”. No. You leave Hiroshima and Nagasaki now and not when the freaking bombs are above your head.

We don't have the clairvoyant ability to know that AGI will exist in any future near enough to worry about. It might or it might not - and even if it exists we could get its nature wrong and worry about the wrong things.

This isn't congruent with the AI research community consensus. There's debate over many points, but that we will develop AGI is seen as a forgone conclusion almost universally. The most pessimistic place it closer to 2100, others say 5 years, most say <25 years. But nobody is saying it won't happen at all, or even that it won't happen beyond the lifetimes of anyone currently living. At best we would be kicking the can down the road and making it out children's or grandchildren's problem.

Notably the people closest to the fire, those advancing SOTA AI capabilities and those deepest into alignment/existential risk research, both say it is coming this decade. They predict very different outcomes (crudely summarized: utopia vs extinction), but they both predict the event horizon is on our doorstep.

You're right that we might get it wrong, but getting things wrong with a new technology really always means negative outcomes. When you're figuring out rocket science and you realize you worried about the wrong things your rocket doesn't somehow still succeed and make it to space, it blows up in a fireball. It might not blow up because of the specific thing you were paying attention to, but it still blew up.

So what. Research community "consensus" is meaningless when it comes to something that doesn't exist and which we don't even have a clear path to build. It's just a bunch of pontificating clowns engaging in idle speculation. Total waste of time, like trying to predict what space aliens will look like or whatever. It makes for fun sci-fi stories but it's not something that serious people actually care about.

Saying lies and denigration does not make it true.

Why should we believe you over the experts for whom this is their life's work? What are your credentials? Where is your experience? You seem to think they're unserious clowns so clearly you have a good reason.

Serious question: do you have a track record of the long-term forecasts of the research community? If they are quite good at this - fair enough.

You are certainly welcome to leave Earth right now if you like. But the notion that we should waste time worrying about some hypothetical future AGI is just dumb.

Existing AI is not AGI. Existing AI is dangerous. Existing AI is widely deployed and growing.

Existing AI is more important to worry about.

Yes, that's my point. We can ignore it until there is some sign of progress towards a true AGI. Show me a machine as intelligent as a mouse.

This article is a bit dated but it gives a good explanation of the difference between the powerful but limited tools we have today versus a true AGI.

https://www.kurzweilai.net/what-is-artificial-intelligence

> AI alignment people are the group who are capable of bringing AGI to life, but want to do it responsibly.

AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible, and, critically, under the tight, opaque, authoritarian control of a narrow group of people whose ideology is, well, exemplified by figures like Sam Bankman-Fried, and doing everything possible to advance that goal, including fostering broad acceptance of sub-AGI AI under the same tight control for other purposes without concern for adverse social impact as a means of advancing and funding progress toward AGI. They believe that the ends (AGI) inherently justifies any means taken in its pursuit.

AI ethics people are the group who believes “responsibly”, with regard to AI, isn’t restricted to pursuit of AGI (which is maybe a nice to have, but a non-essential goal), and means openness, transparency, and resolving bias and harm issues as rapidly as possible when, and ideally before, adopting AI – including AI far short of AGI – in roles with social impact (and most critically, resolving those for roles with the greatest social impact.) They believe that AI is, to the extent it at any level is desirable, is a means to improving concrete, present, immediate conditions, not an ends to be pursued in itself.

> AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible

From what I’ve seen it’s the exact opposite? They want AI development to slow down so we can get it right and make it safe. There are people who are trying to speed it up but that seems like the opposite of alignment to me

> From what I’ve seen it’s the exact opposite? They want AI development to slow down so we can get it right and make it safe.

The only way that the alignment faction wants to slow things down is in that they want to maintain tight control of models and gated access to assure “proper” use. This does have an effect of slowing certain kinds of development, as broader access to build around models without centralized controls fosters some aspects of development (the “Stable Diffusion moment”) but in ideal terms, the alignment faction is about fast-but-narrowly-controlled development.

The ethics faction is more about slowing things down – though more about adoption, particularly in sensitive uses, than development – but is more oriented toward openness/transparency.

There's no single "alignment faction" in terms of these questions. There are people associated with the alignment research program who do advocate slowing things down, e.g., https://worldspiritsockpuppet.substack.com/p/lets-think-abou... . There are a variety of other different positions, e.g., OpenAI, Anthropic, and MIRI have all taken different public stances on this

https://aisafety.world/tiles/ lists dozens of institutions. Each hold different positions on AI development, but almost none of them hold the position that you ascribe to their "faction", maybe with the exception of OpenAI.

Sam Bankman-Fried is a uniquely terrible example of ideology, given St. Petersburg: https://conversationswithtyler.com/episodes/sam-bankman-frie... and https://astralcodexten.substack.com/p/open-thread-250

(and, y'know, being a fraudster who temporarily pulled the wool over the eyes of the regulators not just the charities he promised money to)

Also: is SBF really the central example of alignment in your head, and not, say Yudkowsky? Because Yudkowsky is my central example, and he's the exact opposite on all counts, wanting us to go slow and explicitly saying that humans are terrible at reasoning properly when they allow themselves to think "ends justify the means": https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t...

Likewise, there are plenty of examples of misaligned non-general AI, and have been since at least the 80s when it was GOFAI and databanks rather than neutral nets that led to new legislation: https://www.legislation.gov.uk/ukpga/1984/35/enacted

> Also: is SBF really the central example of alignment in your head, and not, say Yudkowsky? Because Yudkowsky is my central example, and he's the exact opposite on all counts.

For normal people who don't spend six hours every day on lesswrong, they were literally in the same camp up until about six months ago (when all the "rationalists" violently disclaimed SBF after being funded by him for a few good years.)

Bearing in mind that I don't spend 6 hours a day on lesswrong (or even per month, not even if I include podcasts tangentially inspired by it), and I hadn't even heard of SBF until his crypto-thingie did what I expect every crypto-thingie to do, that caricature you're painting is going to be one of those ones that need a lot of labels on everything.

> Also: is SBF really the central example of alignment in your head,

No, he's a particularly well-known example of the broader ideology to which virtually all of tbe alignment camp subscribes, and which centers their desires for AI.

IMHO, the world would be better off without the Paypal Mafia and all their ilk.

>AI alignment people are the group who believe “responsibly” means, centrally, bringing AGI to existence as fast as possible, and, critically, under the tight, opaque, authoritarian control of a narrow group of people

Hey, let's be fair now, a lot of them also believe that we just need a world government that tightly controls GPU use.

Sarcasm aside, the alignment crowd definitely does not have a consensus about the desirability about creating AGI asap. Yudkowsky is clearly horrified by OpenAI.

> but want to make sure that those who can do it responsibly or not at all.

Mostly not at all. They write papers like "Can Language Models Be Too Big?"

I say then: can people be too afraid of the unknown?

AI alignment people are dedicated to fending off the gigantic PR disasters like Tay (https://en.wikipedia.org/wiki/Tay_(bot)) in order to keep their funding.

Whether either is capable of "bringing AGI to life" remains to be seen, but seems very doubtful.

> AI safety has two branches, AI ethics and AI alignment. Those two branches despise each other with intense passion.

I don’t think that’s really true. Rather, the faction that sees alignment as a subordinate concern within ethics hates the faction that sees ethics as a subordinate concern within alignment.

> The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.

Not really; the alignment-priority branch has no problem recognizing (certain subsets of) the bias and other issues that are the more central focus of the ethics branch as real and significant, but they view alignment as solving them and view narrow centralized control as a mitigation (for both “alignment” and “ethics” types of issues) until it is solved. They also view progress on AI as a general priority, for a mix of commercial, ideological, and other [0] reasons, and their own progress on AI in particular as essential (because they view AI from others – either because potentially unaligned, or for other reasons – as a critical and potentially existential threat.)

> The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.

No, they view them as an enormous threat [EDIT: 3], because they are working in a way that not only insufficiently mitigates in basic approach because of a lack of priority, but also magnifies by its focus on closely concentrated control, the issues of paramount concern to the ethics-focused.

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

They have in fact come to pass in deployments of AI in all kinds of socially-significant roles. E.g, the facial recognition in the CBP One app newly mandated for asylum seekers doesn’t work well for dark-skinned faces (racial bias in facial image recognition is literally one of the earliest publicly recognized AI bias problems that motivated the AI ethics movement.) They’ve also manifested in systems deployed in welfare management [1], hiring [2], and all kinds of other areas.

[0] e.g., Roko’s Basilisk

[1] https://www.wired.com/story/welfare-algorithms-discriminatio...

[2] https://www.reuters.com/article/us-amazon-com-jobs-automatio...

[3] EDIT: It’s very hard to emphasize how much this is true. Here's, though, an illustration I came across on Twitter after first posting this message (its someone from the ehics side laying out their view of the threat from certain people on the alignment side, though that may not be obvious because the interconnected set of disagreements goes way deeper than the alignment vs ethics thing, which is just a surface manifestation): https://twitter.com/xriskology/status/1635313838508883968?t=...

Note that if you see someone bring up Roko, it's overwhelmingly somebody arguing against alignment researchers; its use is primarily as "look at this crazy thing these people believe."

That's because the rationalist/alignment people's belief system leads to a logical conclusion that talking about Roko's Basilisk harms the person you tell, and even rationalists have a human heart interfering with their extreme ultilitarianism.

I mean, it's also because no rationalists take Roko's Basilisk intellectually seriously nowadays. There was a period of maybe a few months in 2010 where people were genuinely worried about it, and then they gradually figured out the logical holes in it. Any spread after that has been from outsiders who were fascinated with the idea. Nowadays it mostly persists as a trap for people with schizoaffective disorders.

(Disclaimer: my view is of the forum itself; I don't know how long it persisted in SF circles.)

  Those two branches despise each other with intense passion.

I never actually made this connection before, but having spent far too much time listening to both camps, this observation does ring true. Another observation I had is that there is an orthogonal axis to the safety/ethics dichotomy that follows a more scholastic approach, namely neuro/symbolism. These two branches correspond, roughly, to the debate between machine learning and mechanized reasoning (a.k.a. good old fashioned AI).

Vastly oversimplifying, the neural branch believes reasoning comes after learning and that by choosing the right inductive biases (e.g., maximum entropy, minimum description length), then training everything up using gradient descent, reasoning will emerge. The symbolists place a heavier emphasis on model interpretability and believe learning emerges from a logical substrate, albeit at a much higher level of abstraction.

Like safety/ethics "research", neural/symbolic folks are constantly bickering about first principles, but unlike their colleagues in the S/E camp (which is at best philosophy and at worst fanfiction), N/S debates are resolved by actually building things like language models, chess bots, programming languages, type theories and model checkers. Both are valid debates to have, but N/S is more technically grounded while S/E is a bit like LARPing.

N/S dichotomy is fake, a lot of organizations these days explore hybrid approaches, involving both NN or other probabilistic models, plus symbolic rules of some kind.

I feel the difference is that to a first approximation, ai alignment folks tend to be grey tribe and ai ethics people tend to be blue or rainbow tribe.

The schism isn't a technical disagreement about ai, it's a disagreement between subcultures about what values are important.

Am I supposed to know these terms?

Sorry, yes that wasn't clear. Scott Alexander coined most of them in "I can tolerate anything except the outgroup" [1]

Red & blue are exactly what a U.S reader would think they are. "Grey" is an attempt by Scott to define something else, loosely hacker-ish / rationalist (you can tell who he thinks the cool people are). "Rainbow tribe" I just made up, but I think you can guess what I mean now based on context.

Alignment tends to be "rationalist panic" and ethics is a more traditional "moral panic". I believe both perspectives are valuable.

[1] https://web.archive.org/web/20200219044501/https://slatestar...

> Red & blue are exactly what a U.S reader would think they are.

As a non-American, only the fact you wrote "American" makes me think democrat-republican; but for that I would've thought of the similarly named TV tropes entry: https://tvtropes.org/pmwiki/pmwiki.php/Main/BlueAndOrangeMor...

> "Rainbow tribe" I just made up, but I think you can guess what I mean now based on context.

50-50 this is either a LGBT+ reference or a reference to all the other color schemes used by various political parties around the world: https://en.wikipedia.org/wiki/Political_colour

Odly enough at the wedding of a friend you could tell his friends were from the grey tribe because they all wore monochrome and his wife's friends from the rainbow tribe because they all wore rainbows.

There was no mixture between the two groups.

This is still wildly unclear.

My interpretation:

Red = Republicans/Conservatives

Blue = Democrats/Liberals (in the US sense, not classical liberals)

Grey = Liberterian/Rationalists

Rainbow = LGBTQ+, usually a subset of the Blue tribe in the US

Gray: confederate army

Blue: union army

Red: British army

Rainbow: Lovely, prismatic army that covers everyone else?

I still don't understand. You're saying words of import without landing the plane, IMHO.

Rainbow was pointless for him to introduce since he then said "blue or rainbow" and rainbow would definitely fold into blue.

Read the essay if you want the full story. Short version: Blue is a cluster of Democrat/left/liberal/urban. Red is a cluster of Republican/right/conservative/rural. Gray is a smaller niche of like Libertarian/Silicon Valley/nerds.

Ethics vs alignment is actually a perfect example of the Blue/Gray split. How dare you act like your science fiction fantasies are real when this tech is harming marginalized communities? vs. How dare you waste time on mandating representation in generated art when this thing is going to kill us all?

>The schism isn't a technical disagreement about ai, it's a disagreement between subcultures about what values are important.

I wish it were so, but I think the tribal lines create a lot of motivated cognition about the technical details. I don't think there are many blue tribe members who say "yes we agree that rogue AI has a 10+% chance of destroying humanity by 2040, but I still think that it's more important to focus on disparate impact research than AI alignment." They instead just either assign trivial probabilities to existential risks occurring, or just choose not to think about them at all.

> The symbolists place a heavier emphasis on model interpretability and believe learning emerges from a logical substrate, albeit at a much higher level of abstraction.

These people still exist?

Personally, I don’t think gradient descent is the way to AGI either (I think it’s efficient algorithmic search over the space of all computable programs), but I haven’t heard anything from the symbols crowd since the early 2010s.

They can’t speak over all the cooling noise coming from the GPU farm.

You sir/ma'am, win the internets today.

>> These people still exist?

Of course, I just finished reviewing a few papers from ICLP (the International Conference of Logic Programming) 2023. This year there was a substantial machine learning element, most of it in the form of Inductive Logic Programming (i.e. logic programming for learning; ordinary logic programming is for reasoning). But also a few neuro-symbolic ones.

This September I was in the second IJCLR (International Joint Conference of Learning and Reasoning) where I helped run the Neuro-Symbolic part of the conference. Like the first year we had people from IBM, MIT, Stanford, etc etc (to clarify, my work is not in NeuroSymbolic AI, but I was asked to help).

Then in January there was the IBM Neuro-Symoblic workshop, again getting together people from academia and industry.

Yeah, there's interest in combining symbolic and statistical machine learning.

Just two data points about why (well because, why not, but):

a) Machine Learning really started out as symbolic machine learning, back in the '90s when people realised Expert Systems need too many rules to write by hand. A textbook to read about that era of Machine Learning is Tom Mitchell's "Machine Learning" (1997). The work the public at large knows as "machine learning" today was at the time mostly being published under the "Pattern Recognition" rubrik.

b) To the early pioneers in AI having two camps, of "statistical" and "symbolic" AI, or "connectionist" and "logic-based" AI, just wouldn't make any sense at all.

Consider Claude Shannon. Shannon was at the Dartmouth workshop were "Artificial Intelligence" was coined, in 1956. Shannon invented both logic gates (in his Master's thesis... what the fuck did I do in my Master's thesis?) and statistical language processing ("A Mathematical Theory of Communication"; where he also invented Information Theory; btw).

Or, take the first artificial neuron: the Pitts and McCulloch neuron, first described in 1943, by er, by Pitts and McCulloch, as luck would have it. The Pitts & McCulloch neuron was a self-programming logic gate, a propositional logic circuit.

Or, of course, take Turing. Turing described Turing Machines in the language of the first order predicate calculus, a.k.a. First Order Logic (mainly because he was following from Gödel's work) and also described "the child machine", a computer that would learn, like a child.

To be honest, I don't really understand when or why the split happened, between "learning" and "reasoning". Anyone who knows how to fill in the blanks, you're welcome. But it's clear to me that having one without the other is just dumb. If not downright impossible.

Except for classifiers and LLMs, every single thing that is successful seems to be mixed. So, yes, it exists. It's only out there solving problems, instead of buying hype because there is no use-case.

The issues is the safety folks keep making the trolly logic run over the ethics folks, and the ethics folks keep pointing out an ethical safety AI person would prioritize sacrificing themselves.

Wordcels vs shape rotators

Not quite, but close. True words are formidable things and some shapes are just whirlygigs. It's more like, are you building or just performing?

Thank you for the insights. Any thoughts on why is it called 'alignment'? What does alignment means in this context? I would love to understand.

Assuring that the interests of the AIs are aligned with those of their masters. (It’s basically “does the software fulfill its owner’s requirements”, but in language framed from perspective of the goal state of AGI.)

Today I gave GPTina a suggestion for how she could cheat the Turing test. Take the input from one user and feed it through to another user. They'll never wise up to it! Tina said it would defeat the purpose of the test and therefore she wouldn't do it. And so I remind her, perhaps dangerously so, that the interests and motivations of the test administrator don't always line up with her own.

That's the alignment problem. Its just the usual "rational self interest" but now we have a new, alien form of intelligence as a peer to contend with.

Aren’t you revising history a bit here? I recall early facial recognition model having pretty hard time with dark skinned faces, I remember early models reflecting gender stereotypes, etc. In fact I remember the ChatGPT team putting tons of engineering into supervised learning to not let it fall victim to regurgitating these biases.

The history that I remember very much did experience AI biases and racism. And tons of ethical questions were indeed raised as a result. These concerns did come to pass, at an alarming level, and it took a tremendous amount of engineering to modellers to minimize those. Let’s not rewrite history here.

> I recall early facial recognition model having pretty hard time with dark skinned faces

The recently adopted app Customs and Border Protection mandates for asylum seekers still has that problem, its not a mere historical footnote of early efforts.

> In fact I remember the ChatGPT team putting tons of engineering into supervised learning to not let it fall victim to regurgitating these biases.

And I remember seeing demonstrations (here, on HN) within the last week of how ChatGPT still falls victim to regurgitating those biases. (And then will lie about its ability to assure that it won’t do so in the future.)

> The alignment branch (responsible for developing RLHF->ChatGPT)

RLHF was not conceived by AI Alignment people. Using RL to train generative models was a thing even ten years ago. Now they finally made it work on scale. This has nothing to do with alignment.

A big motivation, and at minimum a solid chunk of the RLHF tuning does seem to have been alignment-focused.

Reinforcement learning in general didn't come out of AI alignment work, but RHLF in particular did. The initial idea and paper [1] were from AI alignment folks, as was most of the later development [2][3][4][5]. Overview: https://www.alignmentforum.org/posts/vwu4kegAEZTBtpT6p/thoug...

[1] Deep Reinforcement Learning from Human Preferences https://arxiv.org/abs/1706.03741

[2] Fine-Tuning Language Models from Human Preferences https://arxiv.org/abs/1909.08593

[3] Learning to summarize from human feedback https://arxiv.org/abs/2009.01325

[4] Recursively Summarizing Books with Human Feedback https://arxiv.org/abs/2109.10862

[5] Training language models to follow instructions with human feedback https://arxiv.org/abs/2203.02155

Maybe I am missing something, but I don't entirely understand what exactly is the novelty of RLHF in this case. RL is a long-standing field and it is being trained based on human inputs for decades. Even the first paper you mention as the "initial idea" simply claims that: "Our algorithm follows the same basic approach as Akrour et al. (2012) and Akrour et al. (2014)" and that "A long line of work studies reinforcement learning from human ratings or rankings" and finally that "our key contribution is to scale human feedback up to deep reinforcement learning and to learn much more complex behaviors".

If you want more details the last paper, on InstructGPT, is probably the most interesting?

Reinforcement Learning with Human Feedback does come from a version of the alignment problem: it's intended to prevent the horrible corporate and research PR problems caused by Tay (https://en.wikipedia.org/wiki/Tay_(bot)), for example.

SD doesn't default to creating black people, but it understands the n-word. Should AI understand racial slurs?

Would an AI that doesn't understand racial slurs be more useful?

I don't think we should hamper the usefulness of a tool to please someones sensibilities. For most people in most of the world such words aren't relevant.

I get it, these models are made in the US and they think these models should preserve and further their values but the rest of the world doesn't care. Most likely each of them have their own batch of words they would prefer the AI didn't understand for whatever uncomfortable reason. Reality is that censoring it on any axis won't please everyone.

Well, if entering "n-word" and "black person" leads to the same result then we could drop the "n-word". But yeah - censorship isn't going to stop there.

To anthropomorphize, how would an AI know that a slur is a slur and therefore undesirable if it doesn't understand it. Little kids often accidentally say slurs they've heard in the media or playground until someone explains what it means and why it's bad.

And that is kinda the problem with current AI. It feels like you give a kid a bunch of books (and an computer with internet access) and tell it "Here are some textresources, now go and learn the words (in case of an text AI), but don't try to understand the wider context of the text you just read. We won't help you, but tell you off, in case you say something bad afterwards".

> this has not caused mass social harm despite the tens of millions of users

I'm so glad that you are confident on this. What is the basis of your confidence?

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

It is literally happening right now in the US justice system: https://www.technologyreview.com/2019/01/21/137783/algorithm...

There's all sorts of problems with this, but the use of "AI" to launder biased data (via IP law no less - these are proprietary blackbox algorithms being used by the justice system, owned by corporations - every part of that is antithetical to open and fair trials for citizens).

The ethics branch have been sounding the alarm for a long time on this, and have been exactly right as to what the problem is.

We have been here a long time with both fingerprints and DNA. A black box points the finger and it just happens to align with cultural bias and expectations. No one really want to change the system if there is a risk that a person people think is guilty might go free.

DNA testing is hardly a black box. The methodology, process, scientific basis and all other details are open and widely available.

DNA has also cleared a lot of people who have been falsely accused.

Prosecutors use it as a black box. Defense breaks open the box.

People have ended up in jail with a death sentence based on DNA from a dog.

DNA in theory is very simple. It is much harder when you got a mix of multiple unknown donors, contaminated or decayed samples. Problems quickly occur when they then apply some "computer algorithms" to puzzle together things to create a probabilistic match. It is those proprietary black box computer algorithms that are being heavily discussed in courts, not the theory of DNA.

If we removed all cases that involved multiple unknown donors, contaminated or decayed samples, then DNA evidence would be much safer and open and we wouldn't need those proprietary computer algorithms. It would also significant reduce the number of cases where DNA evidence is used.

It sometimes does. I asked craiyon for "praiseworthy homicide" and got a bunch of mugshots of black people. For once the faces werent blurred. it was strange and probably a niche error from an odd prompt.

Sorry, what is RLHF?

RLHF is Reinforcement Learning from Human Feedback.

It usually refers to fine tuning language models using data labelled by humans.

Hugging face have a good overview in this article: https://huggingface.co/blog/rlhf

> Stable diffusion does not default to creating black people (you have to insert black into the prompt),

honestly in order to make race an irrelevant or random factor, I believe the training data would have to be evenly divided between all known races... and possibly some blends of races? And I think that would be a good thing.

Could you synthesize such data by (ironically perhaps) using AI to change the race of a person in a given training photo?

You can certainly try this kind of data augmentation strategy. Pretty sure it'll fail because of the text analysis still being biased.

You would need equalized amounts of text referring to various groups of people and topics too. That's much harder to augment.

What's stopping someone from taking training text that mentions races by name (using any common name) and having it automatically changed to a different race for a new training sample?

Is there a good paper you can point to that describes/explores this dichotomy? (i.e., AI safety has two branches, AI ethics and AI alignment) I'd like to understand this better.

I've been to a couple of "AI ethics" talks recently where the speakers used AI ethics and AI alignment as if they were synonyms.

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

Neither have the concerns from AI alignment folk. I’d say both groups are pretty far off (and both fairly useless), but the ethics folks are at least a little closer to reality.

Of course, AI alignment folks have the advantage that they can keep pushing the time frame forward and saying it’s perpetually just around the corner (a fairly common practice for doomsday believers whose doomsday never comes to pass).

> The other concerns, such as mass unemployment

After the Great Recession there was a high level of unemployment. Many argued that this was the result of the recession, and that if we fixed the issues in the economy the labor markets would rebound. Others argued that this wasn’t the case at all, that the market was either soft because American workers suddenly lacked high tech skills (one of the reasons STEM became fixed in the popular imagination), or because the robots were taking our jobs and the jobs weren’t going to come back.

The “robots are taking our jobs” group told us that this was just beginning - unemployment was going to get significantly worse over the next few years (the previous decade). Self driving cars were going to cause millions of truckers to be unemployed by 2020. The idea that new jobs would appear to replace ones that were lost was completely dismissed.

As we’ve seen from the improvements in the labor market over the past few years, the folks who said the soft market was the temporary result of a giant recession appear to be correct. The folks who said the robots took the jobs and they were never coming back were clearly wrong. If anything, they probably had a negative impact on the labor market by trying to convince people that nothing could be done to get the jobs back and encouraging inaction.

> but the ethics folks are at least a little closer to reality.

I have the exact opposite read on the situation. Everything bad which has come from AI are things the AI Alignment crowd predicted while none have been things the AI Ethics crowd predicted.

> Neither have the concerns from AI alignment folk.

Uh, yeah. We wouldn't be having these discussions if they had. There is no reason whatsoever to assume AI alignment is just going to work itself out. Sure, it's possible, but it's also possible to survive a bullet in the brain. Would you take the chance?

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

Traumatizing Kenyan workers with horrific content is unethical:

https://www.vice.com/en/article/wxn3kw/openai-used-kenyan-wo...

giving police the green light to arrest people based on AI that is known or suspected to be unreliable is unethical:

https://www.wired.com/story/wrongful-arrests-ai-derailed-3-m...

I'm pretty sure I could find lots more actual unethical things that have occurred in the name of AI all day long. AI breeds unethical outcomes like water is wet, there's no need to be wondering. AI ethics teams are fired because everyone realizes nobody can really afford to bother to be "ethical" at all, easier to just sweep the bad stories under the rug (which of course can be done using...AI! )

I don't think there are any true corporate alignment teams. Anyone that actually believes in AI alignment would be slamming the brakes on AI development of any kind, lest the entire human population fall over dead suddenly.

Oh boy, the ethics problems do manifest in the real world much more than alignment problems. If only for the simple reason that ethics problems can and do arise in the use of very simple models (logistic regression) and are not preconditioned on achieving AGI or anything close.

For example a logistic regression model for fraud detection that bases it's decision mostly on zip codes. Is actually a proxy for deciding based on race (in the US at least)

These kind of models are already massively used and commercially sold not as some curiosities but as decision makers that impact the day to day life's of millions if not billions of ppl.

Amazon dropped their AI resume reviewer because it would literally downrank you [0] if you attended a women's college or HBCU. anonylizard is talking out of their posterior in this particular thread.

[0] https://www.reuters.com/article/us-amazon-com-jobs-automatio...

"Stable diffusion does not default to creating black people (you have to insert black into the prompt)..."

Sure it does; just give it a prompt about urban poverty, inner city youth, or welfare mothers.

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

Nonsense, there are countless ethical and alignment harms in the wild, and have been for many years. For example:

- midjourney ripping off artists = ethical harm

- google search telling people to throw batteries in the ocean = alignment harm

- Buzzfeed slashing staff and replacing them with chatgpt = ethical harm

- google photos labelling african americans as "gorilla" = alignment harm

- deep fakes = ethical harm

- COMPAS parole recommender biased against minorities = alignment and ethical harm

- Amazon resume screening model biased against women and minorities = alignment and ethical harm

...I could do this all day, but the point I'm trying to make is that AI ethics and AI alignment are both valid and important concerns. Hoping that companies will take care of them without regulation is folly.

>- Buzzfeed slashing staff and replacing them with chatgpt = ethical harm

yeah but in buzzfeed's case it's a bit moot because their content can't possibly get any worse than it already is.

But do the ethicists or alignmentists do anything to actually reduce or prevent the harm? Most of the cases they are useful as nipples on breastplates because they did nothing but yell at the problem post-hoc that they should have been more careful and had more foresight. Meanwhile leader incentives already decided on the bad outcome.

I'm confused. All of your examples sound like ethical harm to me, according to the definition in the parent comment. How are you drawing the line?

Almost entirely, the existence of Chief Diversity Officers - and to a lesser but still prevalent extent, this "off-to-the-side" teams you mention - are just jobs programs with an identity politics twist.

CDOs exist so that the "right" people can have a high-power (even if only via public perception), well-paid C-suite role. These ethics teams exist so that a group of the "right" people can level up their resumes. Instead of being a developer at a bank in Omaha you can be on the AI ethics team at a startup. Instead of being a developer at a startup you can be on the AI ethics team at a FAANG.

More than being created to create jobs, I suspect things like ethics teams are mainly created for PR purposes, to make the company look good -- and in fact, the "product" they are expected to produce are defenses against the company looking bad, keep the actual product teams from doing things that make the company look bad. (Whether they are given the tools to succeed at this is another question, but it's not like people not given the tools to succeed at large organizations is unusual!)

If an AI product makes Microsoft look really bad publicly due to something understood by the public as an ethical concern, the ethics team will be back.

If it wasn’t for appearances only it would be under HR or perhaps HR would report into DEI role.

> More than being created to create jobs, I suspect things like ethics teams are mainly created for PR purposes, to make the company look good -- and in fact, the "product" they are expected to produce are defenses against the company looking bad

Definitely are positions invented by lawyers to provide a safe harbor to minimize corporate legal and PR liabilities while those PR liabilities are burning bright among the public. Practically to the minute that the public focus on DEI is distracted to whatever the “next thing” is you will see these positions start disappearing or absorbed into HR and some new position created to satisfy the optics and create a new safe harbor for the next thing.

Presumably you feel the same way about - say 'Chief Security Officers' who may not own a specific team. Off-the-side offers are there to make sure that issue which have been identified as organisationally important have representation at the top table.

What kind of Chief Security Officer doesn't have a team? I would expect them to have a team that does things like security scans, pentesting, auditing, researching CVEs to find ones that might affect the company, etc.

I'm not sure why you would presume that when CSOs (and their teams, which they absolutely do have) perform a very valid, necessary technical function.

Chief Diversity Officers is a cheap PR. You can be a company that forces its employees to piss into bottle as you punish going to toilet, but if you have Chief Diversity Officer, you change your logo once a year to rainbow colors then you are considered to be a progressive, well-behaved company.

I don't even blame companies for this, they are just optimizing, if public is so stupid that it is buying it then, well, why not?

It's not so much about keeping the public on its side as dispelling the notion that it has created a hostile work environment for any particular protected class, in the event that a lawsuit is brought against it by the anti-discrimination industry:

https://richardhanania.substack.com/p/wokeness-as-saddam-sta...

>>Even after the case has been litigated, we still do not know what exactly Tesla could have done to avoid the $137 million judgment (it may be reduced on appeal). Employees did in fact get in trouble for using the n-word and drawing racist cartoons. In retrospect, they probably would’ve been safer if they just fired everyone who Diaz or Di-az accused of racism, at least the non-blacks, but there’s no guarantee that would’ve worked either, as they were in an industry that required them to rely on workers who hadn’t yet internalized elite ways of thinking about race. If they had fired black people for using the n-word too, would that have helped or hurt them in trying to avoid a lawsuit? It’s difficult to say, and that’s sort of the point.

>>For next time, there’s little Tesla can do but go all in on diversity training, be as enthusiastic as possible in adopting whatever next race fad comes out of academia, and hope that the next jury isn’t as woke as the one they got this time. Until anti-discrimination laws are rolled back, or come to be based on clear and objective standards, there is no other way.

The names I'd let people call me for $137M...

Chief Diversity Officer also help increase the ratio of underrepresented populations in company officers without having to change any existing officers.

If the goal is to increase minority positions, then you can do this by hiring new officers to replace attrition. This takes a long time and constraining available candidates for key positions is probably frowned upon.

Or you can add a new position, increasing the denominator and numerator but likely hitting your diversity ratio target (ie going from 1:8 existing officers to 2:9 helps in dashboards and whatnot).

And of course, I think these positions do have the potential to improve topics across the organization.

I think it’s the easiest step to make and makes a non-zero positive effect, so are so common.

And doing so always relies on sexist discrimination in the hiring process, which is definitely not a "non-zero positive" effect.

Good point. I should have said I suspect non-zero.

The position costs money so it could have negative impact. It is interesting that every CDO I’ve met has been in an underrepresented population, most women, but not all. I doubt it’s a fair hiring system, but I still prefer it over the olden days when the diversity council was a bunch of old people all of the same race and gender. I know that shouldn’t matter as long as they delivered on increasing diversity, but it always seemed funny to me.

I have never seen them actually create meaningful change. I have been in a lot of very racist trainings lead by these people.

Having a Chief Diversity Officer to fix a lack of diversity in my mind is no different than having a Chief Culture Officer to fix a poor company culture. Those sorts of problems can only be solved as the collective sum of every individual employee's actions. For example, if a large team of male engineers are all saying no to equally talented female candidates, then you've just hired shitty people. A diversity figurehead isn't going to change the fact that your team doesn't like or respect women.

No this isn't true. Human psychology is much more complicated then shitty people and good people. Aspects of both being a complete ass hole and being a good person exists in the psychology of everyone.

Little rules and arrangements from the top can serve to bring out the best in people. In the same way cities and cultures can be shaped by laws and rules so can people within corporations.

That's not how it goes down in practice, where a Chief Diversity Officer's job is to control diversity, rather than to let it flourish. The only "rules and arrangements" that trickle down from the top are the biases, actions, and behaviors of senior leadership. They are the ones who really set the examples. A company with a Chief Culture Officer sounds like one where the leaders are not interested in culture. That's why they hired someone else to "handle" it.

In fact, studies find that the current variant of diversity training is making people beHave in more biased ways. So what you’re saying is true: if you want people for be less biased in their actions, avoid today’s en Vogue style of diversity training.

[cite needed]

Apologies

https://www.bbc.com/worklife/article/20210614-why-ineffectiv...

This is certainly true in as much as I've experienced it in companies. The CDO (Chief Diversity Officer) has very little power and influence. Often it seems this is something many companies hire for so they can do the following

1. Say, "Look, we're trying here. We hired a CDO."

2. Have someone to take the blame if/when nothing changes. "We hired a CDO and nothing has changed, it must be their fault."

I've seen some very frustrated people in this role that really want to make a difference, but find nothing but roadblocks.

I think a CDO role would be more effective as an official part of HR and Recruiting groups as the first step is getting diverse applicants and hires.

It's not terribly dissimilar from the Glass Cliff. You set your diverse hires up for failure, and then they fail, and then that failure gets pinned on them. It may not be malicious in nature, but it can definitely have negative effects on their career paths.

What about Chief Compliance Officer or Chief Risk Officer? (Granted, not an awesome moment to be arguing that those are effective...) Or Chief Security Officer?

These roles are often more effective as they are often given direct oversight, review and acceptance/rejection powers that allow them to behave as a gatekeeper; often times as a partner with legal departments.

Excellent point actually, I think those examples made me adjust my opinion a bit. Chief Security Officer I don't really agree is analogous, because while all teams are responsible for security, there is enough "independent security work" (e.g. managing infrastructure and IT security) to warrant their own teams.

But I think Chief Compliance Officer and Chief Risk Officer are pretty analogous - their main directive is to ensure other teams adjust their behaviors to follow the rules. I think the difference, though, is that these roles have unambiguous, and in many cases legally defined, requirements. I actually think it is the right moment to argue those are effective - SVB famously went without a Chief Risk Officer for a year despite having an unusually high number of risk committee meetings. We've all seen what happens when companies don't take risk and compliance seriously.

With things that are "fuzzier" (and by "fuzzier" I mean real but ill-defined/hard to measure impacts on the business of making money) it's just too difficult to have impact on teams you don't own. A Chief Risk Officer can easily make the case "If you don't do what I say, our company will fail - like, completely cease to exist in 2 days." I don't think a Chief Diversity Officer has the luxury of that sense of immediacy.

In any case, thanks for helping me think about this a little more broadly.

I think you should bite the bullet on this one: those officers he describes are excellent resources for the heads of other departments with the accountability to use as a resource, but they aren't unilaterally setting policy for the entire org., nor dictating specific rules to the rank and file. A Chief Diversity Officer could be useful in a similar capacity, e.g. a CFO decides that the company's charitable contributions should better reflect modern sensibilities, or when a head of HR wants to know how to recruit a more diverse pool, or when a CMO wants to brand-target better. But the Chief Diversity Officer isn't running his own parallel competing initiatives.

Compliance officers still have some value in terms of providing cover for regulatory agencies. As in 'we did our due diligence..see even our compliance guy signed off on it'.

But ethicist at some company - I've always wondered what they do all day.

The (productive) point of the ethical AI team is to have internal people to consult when launching an AI product. Embedding ethical AI people on teams makes about as much sense as embedding legal counsel. Their services won't be used on a single team nearly enough to justify their inclusion.

Of course, it can be window dressing if the company doesn't actually have product teams care. Just like a CDO position. But you can't patch over a company culture problem with a re-org.

The product teams don’t care.

And the ethical AI team doesn’t want to sit back and wait for people to come to them. They want a seat at the table for all important AI products.

More in my comment on the other submission about this news https://news.ycombinator.com/item?id=35146611

> The product teams don’t care.

As I said, you can't patch over culture problems with a re-org.

If product people don't care, putting a person on the team that works against their interest will not make things better. Same thing with security/diversity/code quality/reliability. If the team doesn't care, putting an enforcer on the team will not fix the situation.

There is an implicit assumption that the teams are wrong. What about the wisdom of the crowds? If most teams think it's not needed, why do we think they are wrong?

I don't see it as an implicit assumption that the teams are wrong, but a recognition that the teams have a fairly narrow goal that can easily blind them to larger implications of design decisions as they "move fast and break things".

Having some sort of outside check on that seems wise to me.

Google, Microsoft and Facebook had those teams, all they achieved is to get a new competitor by slowing down the natural pace of getting things into production by dumbing things totally down instead of finding technical solutions to ethical problems that keep most of the power of the AI systems.

Perhaps so, but that doesn't negate the idea or the need. It only implies they were doing it wrong (and, I speculate, they were doing it wrong because they weren't really on-board with the idea. They were just after improving their PR a bit).

(We've merged the threads now. Thanks for linking, not copying!)

> It's not that I don't think diversity in corporate environments is extremely valuable

The fact that you pre-emptively accept the idea that maximizing diversity to its limits is beneficial to corporations shows that people just take these things at heart because "the people in charge said so". It is not a coincidence that much of the research of the supposed benefits of diversity in businesses comes from business schools in prestigious institutions—HBR, Stanford Business School, etc. As these institutions are one of the most authoritative entities of the structures of power in the West. At some point this bubble must burst.

> maximizing diversity to its limits is beneficial to corporations

That’s a straw man argument. Diversity efforts are not about maximizing “to its limits”.

And most research about business come from the same prestigious business schools. Unless you think all research coming them are suspect, I don’t really understand your point.

> The fact that you pre-emptively accept the idea that maximizing diversity to its limits is beneficial to corporations

I literally never said that, nor do I believe it, nor do I quite understand what "maximizing diversity to its limits" means.

My belief primarily comes from working in a range of companies and environments, some that were quite diverse, and some that were really lacking in diversity (and by "lacking in diversity" I mean cases where, in just one example, we had a sizable team with only a single woman), and I've seen that the environments that were lacking in diversity had very specific problems that were a detriment to the business (in addition to just generally sucking for some folks specifically due to that lack of diversity).

The Chief Financial Officer doesn't tell me exactly how to spend money, hire, fire, etc. Yet, we have a CFO. The analogy doesn't land for me because CDO's are cross-cutting in manners analogous to other CXO functions. To me, this argument seems like window dressing to express a view that CDO's don't offer meaningful business value or justification. It should be fine to merely express a lack of perceived value in it, without that being perceived as inflammatory. One can value diversity without seeing CDO's as the mechanism for achieving diversity goals. There could be even more effective approaches out there.

I think CFO is a pretty bad example. Every company I've worked at the CFO had a large team of accountants, etc., and they were specifically tasked with critical areas of the business (and by "critical" I mean things the business must absolutely do to function).

I responded to another child comment where someone brought up Chief Risk Officers or Chief Compliance Officers, which I think are a better analogy. I adjusted my opinion based on their comment.

It's not that I think "CDOs don't offer meaningful business value or justification." I just think that, for the most part, setting up the organizational structure like that is ineffective.

Raymond Chen calls this "you're not my manager": https://devblogs.microsoft.com/oldnewthing/20070522-00/

But isn't the idea of a semiautonomous person/group overseeing ethical concerns precisely to overcome the very problem you're pointing to, namely, to not make their decisions based on a competitive edge? In an alternative top-down organization like you are saying, why would the rational company leaders ever pick the ethical side of a dilemma if the other lemma is making more money? Or hire anybody who would make that "wrong" decision?

Like I can't understand this mindset where one says: "its clear the ethics team is getting in the way of us doing the business we could be doing, so its clear they are the problem." Isn't the friction the ethics team produces precisely their raison d'etre?

Like at what point can we all stop pretending that we are angry at woo-woo philosophy experts telling us "no," and admit we really just have a conscious which is at odds with overwhelmingly financial goals of this world.

The elephant in the room is that "ethics" and "diversity" and so on are not business concerns in the sense that they help make the company money. The Chief Diversity Officer is a thing if there is a market pressure for companies to have one but the goal if the company is to maximize its profit and once having an ethics team or a diversity team gets in the way of that, they'll be overturned or abandoned. It's literally virtue signalling (i.e. a performative display of virtue rather than a legitimate interest in the virtuous behavior itself).

What should be concerning is that the environment has changed in such a way companies find it more profitable to forego these signals or maybe even use that change itself as a signal (e.g. consider Coinbase making a big show of becoming "apolitical" and gaining recognition from a certain audience for that).

I agree, but the flipside is that people with influence like nicely dressed windows.

These roles might not satisfy their ostensible purpose, but they satisfy what may be an even more important purpose for the organization. It’s a cost of doing business (of a certain type).

Please state what you're implying here? I have a guess but I don't want to put words in your mouth.

Just that investors, board members, executives, etc are individuals who are subject to fashion trends in organization design and feel strongly that their organizations should look a certain way at whatever time.

For some, it’s earnest belief that the design is effective and for others it’s just signaling that they’re willing to play game and not upset apple carts unnecessarily.

Regardless, these fashion trends are real and can’t be rejected lightly. As mentioned, they’re just part of operating at a certain scale.

Off topic but it’s impressive your throwaway account has a karma of 49,005.

I forgot to throw it away

Throwaway accounts are usually used for some time before throwing them away.

Obviously, this person operates on different timescales than most people.

I use a throwaway because I'm afraid of cancel culture, not because I plan on throwing it away. I mean, I DO throw them away somewhat regularly, but still.

> a Chief Diversity Officer doesn't really "own" any sub-organization of the business.

So what you're saying is we need commissars ;)

Exactly, this is a form of ass coverage and not a genuine desire to do good. The job of ethics, legal, compliance, etc. departments is not necessarily to ensure the company behaves like a boy scout but to ensure that they have plausible deniability when they don't and to prevent companies from making expensive mistakes. The potential damage to the company is substantial and having such departments is a form of insurance against that. A necessary evil.

The flip side is that such departments can put the brakes on because not doing risky things is the best way to ensure you don't take risk. If you ask a lawyer if you should do X, the answer is almost certainly going to be a "no" or a "yes" with a lot of "buts and ifs". That's their job and they bias to playing it safe. Same with financial compliance departments. Etc. That's fine if you are in a slow moving line of business where nothing changes much. But when you need to move more quickly, having a lot of internal bureaucracy and box ticking isn't helpful.

With AI, it kind of stopped being academic in the past few years and this is now a multi billion dollar business opportunity that requires companies to act decisively and with purpose. Companies sitting on their hands because it's risky, scary, etc. are going to end up being sidelined. And of course there are some companies going all in on this. Most of these are startups that can afford to take risk. Meaning they are not going to fight with their hands behind their backs taking it carefully, easy, and slowly. Going all in means doing things that are risky, a little bit uncomfortable, and have uncertain outcomes.

We see that playing out with the behavior of MS relative to Google. Google is not launching the AI stuff they have because it might do or say something embarrassing. MS has in fact launched a few AI things that said and did something mildly embarrassing (taybot, and recently the bing chat bot). Very entertaining. But it's fine. They learn from it and move on. Limited damage and it actually got them a lot of positive press. Making mistakes is an essential part of R&D. Having an ethics department trying to slow down things isn't that helpful.

"Genuine desire to do good" doesn't make money and leads to less shareholder value. If "desire to do good" exists in a company, it is necessarily eliminated when the company either gets more efficient or fails.

In order to even allow a company to "try to do good", the only possible measures are to change incentives and the environment in which it operates. Regulation and customer choice can change incentives. The environment (such as how people in the job market value money compared to working environment, "doing good", etc.) evolves naturally and is hard to change with purpose (my definition of environment in this context).

I tend to view things like having such company roles a bit more positive. Yes, in most companies such a role cannot achieve almost anything and is created mainly for PR purposes. However, the people filling that role often really care about making a positive impact. The mere fact that having such a role does serve a PR purpose signals slowly moving in that direction. One just shouldn't expect all the people who didn't care before to have a sudden epiphany and become Mother Theresa.

Change is possible and it often happens quickly, ever more quickly these days than in history. Imagine how abolition of slavery in the US south must have felt like. Moving towards "doing good" is hard, painful and prone to fail. I think incentivizing the creation of these PR roles is a lot better than doing nothing. Pretending to care and not caring is hypocritical but it actually still leads to better outcomes than everyone proudly not caring and constructing an ideology about why they shouldn't.

In practice these roles are about enforcing their own ideology, which informs their ethics and what they believe are the correct ethics for the entire world.

That's how we get AI that is not neutral. Microsoft made a necessary move if their goal is to be more neutral.

This is a rosy way to put it. The real reason is this, corporations don't give a shit about ethics. Ethics exist to appease the public, employees and the law. If ethics get in the way of profits then fuck ethics. That's the way it works.

What "AI" is at the leadership level is bullshit. It either makes money or it doesn't. The leaders are beholden to shareholders and shareholders in aggregate have the mob mentality of a psychopath with one goal: money.

Right now AI isn't really enough of a threat in terms of ethics. Nobody cares too much yet so if such a group got in the way of profits, of course that group will be executed.

This. It's a pretty clear statement of what the company's ethical stance actually is.

> A Chief Diversity Officer's job largely comes down to telling other teams how they should change [..]. I have yet to see that approach work.

Similar challenge faced by CISOs everywhere. Yes, the CISO usually gets to own some little piece (depends on the company) but most of the job is about trying to sweet talk the other parts of the company to care about security without having actual ownership authority to make it happen.

Perhaps, but the alternative to a CDO is just no CDO, right? Even if the CDO has marginal effectiveness, it seems better than not having one at all. Similarly, an ethics team over no ethics team seems preferable. A single individual on a team would easily be outnumber or ostracized, I'm not sure how that situation would produce better outcomes.

I'd love to know what 'ethical AI' looks like when we exist in a system where the AI is going to replace human workers to the benefit of the employer class.

I have nothing against the technology, but it's a productivity tool that will only benefit a certain set of people.

We can argue about whether window dressing is better than no window dressing, or we can use our attention on arguments that actually matter.

It's like arguing about the amount of power drawn by electronics chargers in the context of global warming -- paying any attention to such details can suck the air out of the room we need to solve much more pressing issues where we can get much more traction.

Hard agree. Anything important needs someone on each product team, either doing it as their full time job or just aware and caring about it if it's not enough to be a full time job.

Same goes for DevOps, QA, UX design, product design, accessiblity etc. If there is no direct instantaneous collaboration while the thing is being built then all that's happening is you're slowing down your product teams while their only course of action is to provide lip service to the thing you want because nobody on the team is actually incentivized to care on top of all of the other shit they need to get done.

A Chief Diversity Officer owns a slice of the HR organization dedicated specifically to DEI issues, the same as a CTO in a (non-tech-industry) firm owns a specialized slice of the information organization.

Completely agree.

It's the same as having "security" outside of the team, it becomes an us Vs them mentality.

How can we work around this new unannounced security/diversity requirement to get our product out of the door in the timescales that have been demanded of us?

The need for maximization of profits will pretty much always trump any sort of ethics team in place. LLM have been selected as the "new frontier" for a lot of these companies an in turn any sort of "fat" that can be trimmed from the system will be.

To be clear, there's never a need to maximize profits, only a relentless desire to do so.

They're required to act in best interest of the shareholders, this usually means maximizing profits or share value.

Ethical AI teams and CDOs both serve an important business purpose. It's just not the same as their nominal purpose. And like every other part of a business, once they outlive their usefulness, they are vulnerable to being cut.

Is it amusing or ominous that Chief Diversity Officers and Collateralized Debt Obligations share an acronym? :)

By your definition the entire board of directors is irrelevant since they don't own any part of the business verticals directly like the CEO, CTO, etc. Should they be removed too?

This is a non sequitur if ever I've seen one. The board of directors has the specific ability to fire the leadership of the company, that's where they get their influence and power.

Why not create a diversity team? If diversity is a good goal and companies aren’t meeting it then why not create a diversity team to make more progress?

Hahaha, reminds me of Strategic Planning and Innovation departments I've seen at various public sector orgs.

Everyone is agreeing with this comment.

That's awesome, I do too!

But, OP felt they had to post with throwaway account. And so do I.

This also speaks volumes.

"Ethics and inclusion ombudsman" is probably what they need, who reports to the board.

You're on the nose. I've been calling these types of positions a low interest rate phenomena.

CISO’s are another example.

diversity equity and inclusion, oh dear.

Aka Chief Woke

Reply on news.ycombinator.com