The NeurIPS 2020 broader impacts experiment
statmodeling.stat.columbia.edu
statmodeling.stat.columbia.edu
This is all about politicizing fundamental research, because it's in a field that's in the spotlight. As work becomes more applied, I can see the logic that scrutiny and consideration of its ethical implications should be increased, but pretending that authors of the kind of fundamental advances presented at a research conference like this need to speculate on the kind of end applications they could underpin is just people with nothing of value to contribute trying to muscle in to a popular venue.
These NeurIPS Broader Impact statements seem a bit different, in that they more explicitly call out ethical considerations.
It's funny though that you ask about hammers & nails. Pure math had a strong tradition of people ending up there because they did not want to do applied math, physics, or engineering that could be weaponized and used to kill people/support oppressive governments/infringe upon basic freedoms. Pure math is not particularly thoughtful about ethics, but many folks ended up there because of their ethical views. The Cold War, World War 2, Vietnam, and French occupation of Algeria had particular ripple effects in pure math because of that.
It would be a shame if ML also became a field people avoided because they didn't want to contribute to evil, in their own view.
Take the longer view: it's easy to think we're the first people ever to think about ethics and science/tech/math, but far from it. How are today's struggles the same or different than Grothendieck's, or Laurent Schwartz's, or Emmy Noether's, or Lagrange's or Kovalevskaya's or...? They all had significant reckonings to contend with.
Both ethics and privacy considerations have recently become pretty regular at Computer Vision and Multimedia Processing conferences.
One very popular object detection model (called YOLO) had the main author recently leaving the field because he got concerned about military applications using his research results.
https://medium.com/syncedreview/yolo-creator-says-he-stopped...
Ethics, however, is a humanities field. People in different political affiliations have widely diverging views on it. It will undoubtedly be used to promote the views of one political affiliation over others. Suppose you need to create a technology that can be used for war in order to better treat cancer? Who gets to choose who lives or dies?
I.e., dealing with ethics concerns and/or ethics committees becomes a huge additional workload in itself, so the research is prioritized to minimize dealing with it.
For example, one might stop a research which might help to treat cancer but dealing with the necessary approvals for patient data makes it unfeasible. Instead you switch to a general purpose target domain where it suddenly (unintentionally) could be used for war instead, but being general purpose it does not need to be approved by the ethics committee..
Cryptography is actually a great example. Gauss called number theory the "queen of mathematics" and several key mathematicians (Hardy) escaped there as they figured it could never, ever be used for political purposes or anything else. And then oops, cryptography comes along and it's built entirely on number theory. You never know.
Humanities people know what they know, and I respect that they've done stuff; but I'm sure not going to bow out of the conversation and hand all the ethics stuff off to them. While some really dig deep, some have no idea what the actual technology can do! There was this long thread recently about Proctorio and McGraw-Hill. Is it right for ML researchers who know about the shittiness of facial recognition to simply say "yeah whatever, do whatever you want to students who are more or less trapped by this system, we won't make a peep"? It's improbably that a NeurIPS paper addendum is going to make a huge difference in that particular problem, but we can 1) practice thinking about these things in preparation for disputes we can take part in, 2) provide ideas and information for journalists, politicians, and humanities folks who'll get involved along the way, 3) develop a habit of at least talking about it.
And last, NeurIPS has so many people/teams submitting that I figured it would be inevitable that more checkboxes appear on the checklist for inclusion -- thinning mechanisms always appear when necessary to slow the flow. If not this, it'd be something else.
Maybe this kind of "push" from conferences will help power a change, but the given the replication crisis (which the author mentions) has been well known for a long time and nothing has really happened on that front, I wouldn't hold my breath.
Here's an example of a disclaimer that shows the authors were acting in good faith (not mine):
>As performing a security analysis against a running election server would raise a number of unacceptable legal and ethical concerns, we instead chose to perform all of our analyses in a "cleanroom" environment, connecting only to our own servers. Special care was taken to ensure that our static and dynamic analysis techniques could never interfere with Voatz or any related services, and we went through great effort so that nothing was intentionally transmitted to Voatz’s servers.
I often see the argument that technology can be good or bad depending on how it's used, and I would agree with this. However, too often the solution to preventing misuse proposed by people is "don't think about it", which doesn't seem like a good one to me.
In other words science has to pass trough political filter of the day. If reality is wrong, unfair or politically incorrect (in our subjective view), we're going to reject reality.
Prof. Pedro Domingos published a great piece on this https://spectator.us/militant-liberals-politicizing-artifici...
AI with no debiasing is a mirror. It's pattern recognition. Techniques for achieving fair and equitable AI are frequently quite similar to "playing god" and do come dangerously close to implementing "rejecting your reality and substituting my own" into the data through sophisticated techniques for social justice reasons. This can elevate the impact of an AI developers actions far above what they may have expected.
For instance, I am shocked that the right wing media has not reported more about the "censorship" inherent to some kinds of word vector debiasing techniques proposed in papers after the famous "man is to computer programmer as woman is to homemaker?" paper within NLP. Usually the undesirable associations (programmer is "male" and homemaker is "female") are due to a representation gap either inherit to reality (far fewer female programmers) or due to failure to properly write about and represent female programmers in corpra (Wikipedia editors should write more articles about female programmers).
One of the techniques proposed to prevent this bias is to simply find the specific rows in your dataset that maximizes the undesirable association and remove these rows. In practice, most of these "undesirable" rows (let's say a row is a wikipedia article) will have no inherit bias what-so-ever, and only get their poor scores because the article is about someone where the editors/authors wrote a lot about their relationship with their dad (using male words) before talking about their programming career. It leaves me with a bad taste in my mouth to remove not-sexist articles to mitigate or remove sexist associations.
Maybe the OP could have critiqued this kind of work rather than their original post and it may have been more constructive...
The fact that many of these technologies could be used to steer a cruise missile? Barely on the radar (heh). Nothing about poverty, religious issues, the global south.. but something vaguely resonant with US culture wars? Sound the alarm!
Any real conception of "ethics" in my mind should be studiously divorced from fashion and group affiliation.
I just read around 20 of the broader impact statements from NeurIPS 2020 papers (at random) and exactly 0 covered such topics. Most covered purely technical issues or concerns. One talked about datasets not representing all world populations. Another talked about the model being used to power weapons. One talked about advertising. One talked about essentially subliminal messaging. Two mentioned adversarial attacks leading to potential life threatening injuries (industrial equipment, traffic systems, etc.).
Look, if you're going to have an objection to something then at least look into it rather than blindly forcing your own preconceived views onto it irrespective of reality.
I would expect most of the papers who have to fill in some impact statement to fill it in with the most anodyne thing possible and get on with their jobs.
I'm all for thinking about ethics, and particularly the 'adversarial attacks on poorly understood NNs' you mentioned resonates with me. That doesn't mean I can't also distrust people who want to claim 'ethics' for their political views.
Please be accurate about academic freedom :)
Can we think about how funny it would be if someone was doing social justice rhetoric with that level of energy?
"well, uhh, you see this is, uhh, problematic i guess, they weren't being an ally. cough. you see, there are a lot of, uh, intersectional and marginalized angles here, it's really complicated. lots of microaggressions. anyways, do the odd questions on page 38, see you wednesday."
My God. Can we not just have an old-fashioned down & dirty fight these days without it being labelled a "cancellation"?
> Credit-card scoring algorithms may reject more qualified applicants in order to ensure that the same number of women and men are accepted.
The entire discussion revolves around what defines "more qualified" and why it cannot simply be the output of an algorithm because it reflects biases in training data and even training methodologies. If you just skip past that entire discussion you may think that there is this nefarious worldview of "injecting bias" but that requires a false or at least simplified premise to begin with.
There is a lot of just-world fallacies behind this sort of thinking. The same that leads to suggest that if "X started a business and made a lot of money" X must be by-definition hard working or talented etc. ignoring aspects of generational wealth, opportunities, fallback options if they failed, etc.
Is it possible to read these papers?
I don't entirely agree with this, and some subjects are moving away from this way of working, but it is still very common in computing.
It's interesting that some Neurips workshops are on OpenReview but Neurips itself is not. I've found great value in being able to read the reviews (both as a reviewer and learning how to do good reviews and as writer)
But most importantly, it seems like there needs to be a fundamental advancement in philosophy (beyond the current state of modernism and postmodernism), if we really want to criticize the negative aspects of technocratic governance. Maybe here's an idea I had for a while: I think there is a deeper lineage to machine learning that nobody is talking about: optimization. (Hot take: isn't what we're all doing with deep learning just optimization to fit large datasets?) I think in order to actually create a meaningful discussion about today's AI, it is crucial to look back at the history of optimization (all the way back into the 30s Soviet, where they tried to use linear programming to their planned economy and failed [2].) I think there needs to be a philosophical framework capable of articulating the effects of optimization technology on our society, before we can go any further and tackle the issues with today's and tomorrow's AI.
[1] https://en.wikipedia.org/wiki/Phrenology [2] https://crookedtimber.org/2012/05/30/in-soviet-union-optimiz...
NeurIPS is an extraordinarily competitive conference. Most authors will look at it as another way to potentially be rejected by a reviewer/area chair (in spite of all assurances about the process), and will write something incredibly trite and bland to make sure nobody is upset.
Another concern is that this leads to a death by a thousand cuts - it's very easy to justify asking authors to write an additional section - but - if someone with sufficient authority doesn't say no, the logical endpoint is something like the application process for tenure track positions, where you need to produce something in the order of hundreds of pages (teaching statement, research statement, cover letter, diversity statement, list of funding, recommendation letters, etc).
I see exponentially stronger algorithms every year, but the advancement in being able to set a limit to it is nothing so far.
You do? I don't. My sense as somebody near (but not in) the ML research community is that, while there have been recent flashy new things like GPT3 or the new protein folding benchmark, these are more like "solid improvements to applications of things we mostly already know". Are you referring to something else?
Also many people forget that we have working self driving cars on the road, 20 years ago nobody thought that it would happen so fast.
Nothing done by humans is objective. Objective systems don't, for example, have massive reproducibility problems.
This seems like a hyperbole. Obviously there is a lot of garbage that people claim to be science. But 2+2=4 is subjective?
Basically one thing I love about math is you can take any statement/idea/construction, consider some opposite of it, and see if you can get something coherent out -- and often enough you do! I'm not saying that 2+2=4 or "you can't differentiate discontinuous functions" is subjective, per se, but it is in some sense a decision, a choice.
Well Google, there you go.
The example I gave is perhaps the worst possible scenario, where AI systems are specifically engineered for the goal of racial bias. The recent advancements in facial recognition technology (helped by the great progress in image recognition by neural nets) make possible a wide-spread surveillance system. I would certainly call the scale enabled by the automatable human-level performance of facial recognition systems dangerous, just like I would call the scale of destruction of nuclear weapons dangerous, even if they are dangerous in different ways.
Even then, e.g. in the United States AI systems aren't necessarily engineered for bias. However, if we're not careful, existing racial, gender, or socioeconomic biases in society can enter these systems and can be reinforced by them (this has already happened in well-documented ways). There I think AI ethics researchers can provide immense value in helping us identify and fix these urgent problems, and I welcome ethics researchers playing a part in any field for this reason.
I kid, but only kind of.