One of these can be objectively evaluated for skill. There may be unwanted side effects. But the skill is there or it isn’t. That does not apply to the other, which makes their hiring and firing fundamentally different.
https://facctconference.org/2021/acceptedpapers.html
it's a mix of "soft" humanities, applied ML, methodology, algorithms/theory, and HCI.
it's not any harder to evaluate an AI ethics researcher than it is any other kind of researcher.
With AI ethicists... if you could look 100 years in the future.. there's not an objective measure of ethical outcomes you could use to evaluate the performance of your AI Ethics team. I suppose a cynic would look at the total cost spent defending and paying settlements / judgements in ethics-related lawsuits. As far as a non-cynical objective metric for AI Ethics performance, I can't think of any.
You could also look at compliance costs?
> dollars lost due to losing trust.
Trust, ethics, and public moral outrage are hopefully related, but not the same thing. Also, measuring loss of trust and allocating changes in revenue to that change is very difficult.
> compliance costs?
I think many people would agree that there's often a big difference between legal requirements and ethics.
If I understand what you saying correctly, you're saying AI ethics researchers have no skill?
It is far easier to evaluate a security researcher's output than an ethics researcher.
I think many of us are familiar with this concept, as many of us likely preferred math over english in school as we preferred being deterministically evaluated.
I don't think OP is claiming that there ethicists have no skill, but rather that it's fundamentally a lot more difficult to evaluate.
However they're empowered to fix them, essentially no questions asked. So you don't often get p0 posts about unfixed bugs in Google products.
It is instead related much more closely to a person's sense of right and wrong.
Of course AI ethics researchers have skills. They have skill in researching, they have knowledge of AI, and presumably education in formal logic and ethics. They have skill in writing and communicating the results of their findings.
However the product they produce is not writing and it's not research. It's the commentary on the research, the recommendations based on the results they have found. It's very subjective.
It's very different than a skilled hacker applying their skill and knowledge to uncover a zero-day exploit in your software that needs to be fixed. That's a much more objective process.