Microsoft lays off one of its responsible AI teams

Isn’t it a bit early to tell? I can foresee ChatGPT being used to farm “karma” on social media sites to bot accounts to credibility on sites like Reddit - maybe even HN.

AI ethics people are the group who are not capable of bringing AGI to life, but want to make sure that those who can do it responsibly or not at all.

I don’t think that’s really true. Rather, the faction that sees alignment as a subordinate concern within ethics hates the faction that sees ethics as a subordinate concern within alignment.

> The alignment branch (responsible for developing RLHF->ChatGPT) believes AI ethics is completely trivial, and distracts attention away from existential risks.

Not really; the alignment-priority branch has no problem recognizing (certain subsets of) the bias and other issues that are the more central focus of the ethics branch as real and significant, but they view alignment as solving them and view narrow centralized control as a mitigation (for both “alignment” and “ethics” types of issues) until it is solved. They also view progress on AI as a general priority, for a mix of commercial, ideological, and other [0] reasons, and their own progress on AI in particular as essential (because they view AI from others – either because potentially unaligned, or for other reasons – as a critical and potentially existential threat.)

> The ethics branch (responsible for all the media attention about racist AIs/bias etc) view alignment people as pie in the sky researchers whose results are irrelevant to society.

No, they view them as an enormous threat [EDIT: 3], because they are working in a way that not only insufficiently mitigates in basic approach because of a lack of priority, but also magnifies by its focus on closely concentrated control, the issues of paramount concern to the ethics-focused.

> It seems increasingly that the concerns from the ethics branch have not come to pass at all.

They have in fact come to pass in deployments of AI in all kinds of socially-significant roles. E.g, the facial recognition in the CBP One app newly mandated for asylum seekers doesn’t work well for dark-skinned faces (racial bias in facial image recognition is literally one of the earliest publicly recognized AI bias problems that motivated the AI ethics movement.) They’ve also manifested in systems deployed in welfare management [1], hiring [2], and all kinds of other areas.

[0] e.g., Roko’s Basilisk

[1] https://www.wired.com/story/welfare-algorithms-discriminatio...

[2] https://www.reuters.com/article/us-amazon-com-jobs-automatio...

[3] EDIT: It’s very hard to emphasize how much this is true. Here's, though, an illustration I came across on Twitter after first posting this message (its someone from the ehics side laying out their view of the threat from certain people on the alignment side, though that may not be obvious because the interconnected set of disagreements goes way deeper than the alignment vs ethics thing, which is just a surface manifestation): https://twitter.com/xriskology/status/1635313838508883968?t=...

  Those two branches despise each other with intense passion.

Vastly oversimplifying, the neural branch believes reasoning comes after learning and that by choosing the right inductive biases (e.g., maximum entropy, minimum description length), then training everything up using gradient descent, reasoning will emerge. The symbolists place a heavier emphasis on model interpretability and believe learning emerges from a logical substrate, albeit at a much higher level of abstraction.

Like safety/ethics "research", neural/symbolic folks are constantly bickering about first principles, but unlike their colleagues in the S/E camp (which is at best philosophy and at worst fanfiction), N/S debates are resolved by actually building things like language models, chess bots, programming languages, type theories and model checkers. Both are valid debates to have, but N/S is more technically grounded while S/E is a bit like LARPing.

The history that I remember very much did experience AI biases and racism. And tons of ethical questions were indeed raised as a result. These concerns did come to pass, at an alarming level, and it took a tremendous amount of engineering to modellers to minimize those. Let’s not rewrite history here.

RLHF was not conceived by AI Alignment people. Using RL to train generative models was a thing even ten years ago. Now they finally made it work on scale. This has nothing to do with alignment.

I'm so glad that you are confident on this. What is the basis of your confidence?

It is literally happening right now in the US justice system: https://www.technologyreview.com/2019/01/21/137783/algorithm...

There's all sorts of problems with this, but the use of "AI" to launder biased data (via IP law no less - these are proprietary blackbox algorithms being used by the justice system, owned by corporations - every part of that is antithetical to open and fair trials for citizens).

The ethics branch have been sounding the alarm for a long time on this, and have been exactly right as to what the problem is.

honestly in order to make race an irrelevant or random factor, I believe the training data would have to be evenly divided between all known races... and possibly some blends of races? And I think that would be a good thing.

Could you synthesize such data by (ironically perhaps) using AI to change the race of a person in a given training photo?

I've been to a couple of "AI ethics" talks recently where the speakers used AI ethics and AI alignment as if they were synonyms.

Neither have the concerns from AI alignment folk. I’d say both groups are pretty far off (and both fairly useless), but the ethics folks are at least a little closer to reality.

Of course, AI alignment folks have the advantage that they can keep pushing the time frame forward and saying it’s perpetually just around the corner (a fairly common practice for doomsday believers whose doomsday never comes to pass).

> The other concerns, such as mass unemployment

After the Great Recession there was a high level of unemployment. Many argued that this was the result of the recession, and that if we fixed the issues in the economy the labor markets would rebound. Others argued that this wasn’t the case at all, that the market was either soft because American workers suddenly lacked high tech skills (one of the reasons STEM became fixed in the popular imagination), or because the robots were taking our jobs and the jobs weren’t going to come back.

The “robots are taking our jobs” group told us that this was just beginning - unemployment was going to get significantly worse over the next few years (the previous decade). Self driving cars were going to cause millions of truckers to be unemployed by 2020. The idea that new jobs would appear to replace ones that were lost was completely dismissed.

As we’ve seen from the improvements in the labor market over the past few years, the folks who said the soft market was the temporary result of a giant recession appear to be correct. The folks who said the robots took the jobs and they were never coming back were clearly wrong. If anything, they probably had a negative impact on the labor market by trying to convince people that nothing could be done to get the jobs back and encouraging inaction.

Traumatizing Kenyan workers with horrific content is unethical:

https://www.vice.com/en/article/wxn3kw/openai-used-kenyan-wo...

giving police the green light to arrest people based on AI that is known or suspected to be unreliable is unethical:

https://www.wired.com/story/wrongful-arrests-ai-derailed-3-m...

I'm pretty sure I could find lots more actual unethical things that have occurred in the name of AI all day long. AI breeds unethical outcomes like water is wet, there's no need to be wondering. AI ethics teams are fired because everyone realizes nobody can really afford to bother to be "ethical" at all, easier to just sweep the bad stories under the rug (which of course can be done using...AI! )

For example a logistic regression model for fraud detection that bases it's decision mostly on zip codes. Is actually a proxy for deciding based on race (in the US at least)

These kind of models are already massively used and commercially sold not as some curiosities but as decision makers that impact the day to day life's of millions if not billions of ppl.

Sure it does; just give it a prompt about urban poverty, inner city youth, or welfare mothers.

Nonsense, there are countless ethical and alignment harms in the wild, and have been for many years. For example:

- midjourney ripping off artists = ethical harm

- google search telling people to throw batteries in the ocean = alignment harm

- Buzzfeed slashing staff and replacing them with chatgpt = ethical harm

- google photos labelling african americans as "gorilla" = alignment harm

- deep fakes = ethical harm

- COMPAS parole recommender biased against minorities = alignment and ethical harm

- Amazon resume screening model biased against women and minorities = alignment and ethical harm

...I could do this all day, but the point I'm trying to make is that AI ethics and AI alignment are both valid and important concerns. Hoping that companies will take care of them without regulation is folly.

But I think Chief Compliance Officer and Chief Risk Officer are pretty analogous - their main directive is to ensure other teams adjust their behaviors to follow the rules. I think the difference, though, is that these roles have unambiguous, and in many cases legally defined, requirements. I actually think it is the right moment to argue those are effective - SVB famously went without a Chief Risk Officer for a year despite having an unusually high number of risk committee meetings. We've all seen what happens when companies don't take risk and compliance seriously.

With things that are "fuzzier" (and by "fuzzier" I mean real but ill-defined/hard to measure impacts on the business of making money) it's just too difficult to have impact on teams you don't own. A Chief Risk Officer can easily make the case "If you don't do what I say, our company will fail - like, completely cease to exist in 2 days." I don't think a Chief Diversity Officer has the luxury of that sense of immediacy.

In any case, thanks for helping me think about this a little more broadly.

But ethicist at some company - I've always wondered what they do all day.

Obviously, this person operates on different timescales than most people.

I responded to another child comment where someone brought up Chief Risk Officers or Chief Compliance Officers, which I think are a better analogy. I adjusted my opinion based on their comment.

It's not that I think "CDOs don't offer meaningful business value or justification." I just think that, for the most part, setting up the organizational structure like that is ineffective.

And the ethical AI team doesn’t want to sit back and wait for people to come to them. They want a seat at the table for all important AI products.

More in my comment on the other submission about this news https://news.ycombinator.com/item?id=35146611

Little rules and arrangements from the top can serve to bring out the best in people. In the same way cities and cultures can be shaped by laws and rules so can people within corporations.

In order to even allow a company to "try to do good", the only possible measures are to change incentives and the environment in which it operates. Regulation and customer choice can change incentives. The environment (such as how people in the job market value money compared to working environment, "doing good", etc.) evolves naturally and is hard to change with purpose (my definition of environment in this context).

I tend to view things like having such company roles a bit more positive. Yes, in most companies such a role cannot achieve almost anything and is created mainly for PR purposes. However, the people filling that role often really care about making a positive impact. The mere fact that having such a role does serve a PR purpose signals slowly moving in that direction. One just shouldn't expect all the people who didn't care before to have a sudden epiphany and become Mother Theresa.

Change is possible and it often happens quickly, ever more quickly these days than in history. Imagine how abolition of slavery in the US south must have felt like. Moving towards "doing good" is hard, painful and prone to fail. I think incentivizing the creation of these PR roles is a lot better than doing nothing. Pretending to care and not caring is hypocritical but it actually still leads to better outcomes than everyone proudly not caring and constructing an ideology about why they shouldn't.

https://richardhanania.substack.com/p/wokeness-as-saddam-sta...

>>Even after the case has been litigated, we still do not know what exactly Tesla could have done to avoid the $137 million judgment (it may be reduced on appeal). Employees did in fact get in trouble for using the n-word and drawing racist cartoons. In retrospect, they probably would’ve been safer if they just fired everyone who Diaz or Di-az accused of racism, at least the non-blacks, but there’s no guarantee that would’ve worked either, as they were in an industry that required them to rely on workers who hadn’t yet internalized elite ways of thinking about race. If they had fired black people for using the n-word too, would that have helped or hurt them in trying to avoid a lawsuit? It’s difficult to say, and that’s sort of the point.

>>For next time, there’s little Tesla can do but go all in on diversity training, be as enthusiastic as possible in adopting whatever next race fad comes out of academia, and hope that the next jury isn’t as woke as the one they got this time. Until anti-discrimination laws are rolled back, or come to be based on clear and objective standards, there is no other way.

If the goal is to increase minority positions, then you can do this by hiring new officers to replace attrition. This takes a long time and constraining available candidates for key positions is probably frowned upon.

Or you can add a new position, increasing the denominator and numerator but likely hitting your diversity ratio target (ie going from 1:8 existing officers to 2:9 helps in dashboards and whatnot).

And of course, I think these positions do have the potential to improve topics across the organization.

I think it’s the easiest step to make and makes a non-zero positive effect, so are so common.

If an AI product makes Microsoft look really bad publicly due to something understood by the public as an ethical concern, the ethics team will be back.

That’s a straw man argument. Diversity efforts are not about maximizing “to its limits”.

And most research about business come from the same prestigious business schools. Unless you think all research coming them are suspect, I don’t really understand your point.

I literally never said that, nor do I believe it, nor do I quite understand what "maximizing diversity to its limits" means.

My belief primarily comes from working in a range of companies and environments, some that were quite diverse, and some that were really lacking in diversity (and by "lacking in diversity" I mean cases where, in just one example, we had a sizable team with only a single woman), and I've seen that the environments that were lacking in diversity had very specific problems that were a detriment to the business (in addition to just generally sucking for some folks specifically due to that lack of diversity).

It's like arguing about the amount of power drawn by electronics chargers in the context of global warming -- paying any attention to such details can suck the air out of the room we need to solve much more pressing issues where we can get much more traction.

That's not a question of ethics, it's a question of the model performance.

Who is bullying who exactly?

>who exactly authorised you to project legitimacy in a space that has not seen any regulation?

He's using the second meaning of 'legitimate'

   able to be defended with logic or justification; valid.

Why is it that in a forum skewed libertarian techbros, there's this nigh-religious faith that corporations will not only get AI right, but get its impacts on society right? You know, with their stellar track record and all.

To me this is just another of countless demonstrations of silicon valley ego/hubris.

What do you mean by this?

Many people throw out the word ‘theoretical’ as some kind of way to imply that something ‘isn’t real’ or worth worrying about. Something might seem implausible until it happens. Gravity waves were once ‘only’ theoretical after all :)

There are plenty of AI dystopian predictions, and many of these are possible and impactful. Many of these theories are based on solid understandings of human nature. It is hard to know how technology will evolve, but we have some useful tools to make risk assessments.

> The dangers of corporations being in control aren’t.

There have been plenty of dystopian predictions about corporate control too. I take the point that we’ve seen them over and over throughout history.

This is a false dichotomy. Regulation can take many forms. It is an essential tool to reduce the probability and impact of market failures.

Using medical/health metaphors can help frame these discussions in less polarizing ways:

1. How do we balance prevention versus treatment? The key question is not if particular markets can struggle and fail in certain conditions. They do. More insightful questions are: (a) how should we mitigate the consequences and (b) what kinds of regulation are worth the cost?

2. What about public health? What happens when the patient won’t get vaccinated and contagion spreads? / We see this in computer security. Lax security by one can spillover to many. / Banks (and even financial systems), left to their own devices are not as resilient and fair as we would like.

Underlying this whole discussion is also “what timeframe are we optimizing for?” and “what exactly are we optimizing for? Economic efficiency? Equality? Something else? Some combination?”

I don't think that the most successful average-user-facing AI product is going to be the least self-regulated, though. Those restrictions don't turn away typical users wanting quick answers, just people who are asking political questions or want to mess around seeing what they can make the AI say.

Further, silicon-based lifeforms would be able to spread and travel the universe much easier because what is time to a machine?

For a brief glorious moment in time, we created a lot of value for shareholders.

Sure, regulation would help here but your examples are of regulations that have existed for absurdly long periods of time and would not be able to be implemented in todays society from scratch.

It’s nearly impossible to take something away. People get accustomed to a certain way of doing it and disruption just means political upheaval when we’re talking about something being banned by the government.

As a hypothetical, think about applying the same laws as in the Middle East to US citizens and how likely it is that anyone would wear a head scarf that wasn’t already doing so.

How are you measuring intelligence? How are you defining intelligence? Formal measures of intelligence like ARC still show even SOTA LLM’s doing incredibly poorly.

As Chollet said, people are misinterpreting LLMs as looking or seeming intelligent, therefore they are, just in the same sense that they think there must be lemon in this lemon flavored candy since it tastes like lemon, when there’s no lemon at all.

This just seems like more Yudkowsky BS about how AGI is right around the corner, despite industry leaders (Ng, Chollet, etc.) saying we aren’t much, if any closer to AGI despite recent developments. There’s no indications that LLM -> AGI.

I really, really have got to stop coming here.

> Oh, you can't hire enough women, you need to hire me again!

There it is. You didn't need to blow the dog whistle, we all got it. Keep on keepin' on ignoring the ever present boys' club and widespread reports that diversified teams produce better results. Surely that's relevant to a discussion about... an internal AI ethics team. Yup.

No.

1. Even if you're not going to do any mitigation, you do need engineers and researchers to help do the risk analysis. It's a collaborative process; rarely can a lawyer provide a full risk analysis without asking questions of eng/research, and sometimes answering those questions requires doing novel R&D and/or non-trivial engineering work

2. What happens when the lawyers say "very risky" and management isn't willing to take that risk but also doesn't want to "sit out"? In that case, technical folks are needed who can help mitigate risks. Constraining and aligning the output of these models is non-trivial and in many cases remains a completely open research problem.

And that's only the legal risk. As my comment pointed out, there are also functional requirements. Consider a chip maker fabricating a new chip: eliminating bugs in the architecture demands hardware V&V, regardless of what legal determines about liability, because the chip doing what you want it to do is a core product functionality. Similar for most applications of text generation models (image less so, for now, but any integration with the real economy will introduce the same sort of core product functionality requirements).