Hacker News Reader

I’m not sure enough people actually read the article, though. Here’s what the person proposing this change said:

> In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention.

Statements like that are frightening to me. It’s incredibly hard to define “negative“ in a reasonable way, and I guarantee you that someone else is going to define it in a way you don’t like.

What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that?

Seems like Nature is encouraging modern-day Ludditism. We have a historical perspective to reject this out of hand: workers are now much better off than in the pre-industrial period. And yes, I understand that the past does not predict the future, but that cuts both ways. As a rule doom and gloom predictions do not age well.

> What’s the negative impact of the recent discovery of a non-quantum recommendation algorithm on the field of quantum computing? Should the young researcher have spent time considering and detailing that?

If we consider the goal of research to improve society, then yes, I think it's reasonable to expect a researcher to spend some time considering whether what they're doing is going to actually improve society. In biology, most funding agencies require you to take a brief (~8-10 hours of class time) course on the responsible conduct of research, even if you aren't doing any human subjects research.

In the specific case you gave of the non-quantum recommendation algorithm, the negative impacts should be more or less the same as the impacts of the quantum algorithm, but with less of a barrier to entry. And even the person proposing this acknowledges: "We all agree it’s hard and that we’re going to miss tonnes of them, but even if we catch just 1% or 5%, it’s worth it."

I think in the case mentioned, the implication is that advertisers would be better equipped to target us with ads – which is a fair point.

But why must we burden the theorists with ethics, and not the programmers or companies who bring that theory into the real world?

Why is that statement so frightening to you?

Because it highlights exactly how broad the definition of “negative” can be. I grew up in a family of blue collar workers. My uncle worked at steel mill that shut down. Perhaps with some automation it could have at least stayed open. Regardless, he eventually found a new rewarding job.

My dad is out of a job now, too, but that’s because he got hurt at work, multiple times. He has had back issues my entire life because he got hurt at work.

The person proposing a consideration of negative consequences was “inspired” by jobs being automated away. Whether a researcher has considered the negative consequences enough will be determined by peer reviewers who have their own political leanings and potential agendas. In the end, I would expect the process to become completely subverted, without accomplishing much along the way.

It seems quite strong to expect "researchers disclose any possible negative societal consequences of their work in papers".

This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking).

The verbiage around fully disclosing the possible outcomes adds an extra wrinkle. Reasonable results require social and technological forecasting and weighing of what researchers thing is likely or reasonable. It's possible that the researchers we're discussing might not have extensive training in forecasting large-scale social behavior.

I would submit that it's worth this effort to catch some small percent of potential negative impacts only if we can do so without excessive costs from false positives. A scenario in which an excess of false positives overwhelms the number of true positives at great cost is a very possible outcome and one that merits more than casual dismissal. If we can't predict outcomes reliably and can't agree on how to weigh them, then some might consider it wise to not predicate significant and consequential decisions based on a combination thereof.

You're absolutely right. Researchers should be taking a look at the long-term consequences of their work. It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions.

Hecht's heart - and yours - are absolutely in the right place. We can, and should, think carefully and deeply about the consequences of all of our choices with compassion and humility.

> This isn't just a matter of knowledge and familiarity. It requires some intensely personal moral and ethical judgments. One person's moral evil (self-driving trucks putting truckers out of work) might be another's moral obligation (getting rid of the life-shortening occupational hazards of trucking).

Absolutely. We shouldn't expect the authors of one paper to decide on the utility of various consequences and alternatives. Also, those consequences don't exist in a vacuum, and depend on other assumptions. Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive. And this provides an opportunity to kick off other lines of research and inquiry.

> ...excessive costs from false positives...

Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article)

> It's just possible that researchers should not assume that they're equipped to make long-term predictions or shape the efforts of their peers around said predictions.

I think excuses based on insufficient expertise is a cop-out. There is no social science specialization with predictive models such things. The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society. If they can vote, they can damn well put some thought into possible consequences.

--

The way I see it, there are two goals underlying this suggestion:

1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth)

2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve.

Most commenters are focusing on only the first one; I think the second is equally important. If the focus was mainly on the first problem, we might choose to have editors writing editorials every few months on recent advancements in some topic. Or a conference panel discussing the same. Those will help towards the first goal, but only partially towards the second goal, since most researchers could "outsource" ethics to others.

> Eg: Automating jobs is possibly an ethical positive provided people don't need jobs to survive.

Or maybe automating certain jobs is an ethical positive because means fewer people die in coal pits. You're very right that this is an excellent opportunity to kick off other lines of research and inquiry. It's possible that what is essentially a journal of applied mathematics might not be an ideal venue for such, given that journals of ethics and economics already exist.

> Could someone give concrete examples of the possible costs of "false positives"? In the picture I have in mind, papers are not accepted/rejected based on whether someone likes the consequences or not. (as stated in the article)

What would be the point of a negative social consequences disclosure requirement if there were no consequences whatsoever for leaving one off or being glaringly silly in the disclosure? The point of this proposed process is that it is ground to reject a paper and require further revision.

To put it another way: the process proposed will either have false positives or it will do nothing. The extent to which it has false positives that exact a cost will be determined by the extent to which the process works as intended to discourage research deemed to have negative social consequences.

Rejecting a paper might end a grad student's career. It might prevent disclosure of a significant finding in time to have a major impact on something important. We have no way of anticipating the consequences of this. Perhaps this proposal should come with a discussion of the potential negative social consequences of it, given that some can already be reasonably foreseen.

Given how abstract that is, I shall offer a more concrete example. My past involves research into how to make circuits more efficient and easily produced reliably and at scale. At no point in this process did anyone stop and think about if this might have applications in computers used as part of weapons systems. Which, looking back, is obviously one application of computers.

> The authors are expected to have the most insight into the specific aspects they are writing on, and generally educated well enough to be a functioning citizen of modern society.

You're absolutely right. The authors can be expected to have deep and significant insight into the technical or mathematical aspects of their work. It's possible that this might not be the same as having the most insight into economic or social consequences of their work.

> 1. Get a better understanding of the possible consequences of different lines of advancement, and, as a community, encourage the "positive" ones while discouraging the "negative" ones. (for some value of positive/negative, which might well be emergent only after much back-and-forth)

This is a laudable goal! And it's perhaps worth considering if there might be unintentional and perhaps "negative" side effects to such a process.

> 2. Get each and every researcher to actively think about the ethics of their research and the problems they choose to solve.

If this is the goal, then perhaps the requirement should be for every paper to be accompanied with an ethical statement which is in no way part of the review process. That would satisfy your needs without risking significant unintentional side-effects, would it not?

Since the review process is iterative, the reviewer will point out what they consider to be a shortcoming, and the author will fix that part. If they disagree, and the author is keen on not mentioning those concerns even as worth considering, but the paper is otherwise good, we could evolve a system whereby the reviewer comment is published along with the paper.

> If this is the goal, then perhaps the requirement should be for every paper to be accompanied with an ethical statement which is in no way part of the review process. That would satisfy your needs without risking significant unintentional side-effects, would it not?

How do we stop this ethics statement from being vacuous, unless it is reviewed? It might quickly devolve along the lines of TOS/EULA (legalese to include all edge cases) or reduce to something trivial like: No fingers were harmed while typing up this paper.

> Since the review process is iterative, the reviewer will point out what they consider to be a shortcoming, and the author will fix that part.

That takes a non-zero amount of time, and thus funding. So now we've got a system where disagreeing about someone's forecasting of potential social consequences or their decisions around ethics imposes costs on mathematics.

Even processes intentionally and explicitly designed to be iterative have an unfortunate tendency to stop iterating as conditions change. This adds one more reason for papers to not reach publication.

> If they disagree, and the author is keen on not mentioning those concerns even as worth considering, but the paper is otherwise good, we could evolve a system whereby the reviewer comment is published along with the paper.

That sounds like a good idea. In fact, why not just skip the review portion of it and publish reviewer's comments on a paper's ethics along side it every single time? It would seem to get to the same point more quickly!

> How do we stop this ethics statement from being vacuous, unless it is reviewed?

Here we have the crux of it: the whole point of this proposal is for there to be real consequences attached. Which brings us back to my point about false positives and side effects.

I suspect that in most cases, the statements would correctly be vacuous. Many - probably most - papers in computer science have minimal to no obvious or reasonably forseeable social implications.

> EDIT: Most papers/presentations already talk about the possible positive impacts -- nobody holds back on that for lack of expertise.

That's not really accurate. Sure, people talk about positive impacts - but in a technical CS paper, they talk about positive technical aspects, e.g. "this new method will allow you to train ML models faster". They don't usually talk about societal impacts, which is what the article is about.

> Firstly, this is for researchers/academics who are publishing in journals, disseminating ideas for new technology.

The implicit assumption is the engineers will use it to make something bad. The scientists must anticipate what engineers will do and document potential negative actions. At worst its useless boilerplate, at best its advertising all the negative uses. Science and Engineering are inextricably linked when it comes to impacts on society.

The next step of course is to stop research if its determined someone can do something bad enough with it. Why else would you go through the effort of documenting it?