Hacker News Reader

Comment by slededit | Hacker News Reader

I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredibly patronizing.

Engineers are not philosophers and should not be placed in this role. We do not have the tools to do it. As with any human we have a duty to consider first order effects - but even the most astute philosopher will have trouble anticipating second order effects. This burden should not be placed on those least qualified to perform it.

Edit: Thank you to everyone for your thoughtful replies. I find this topic fascinating to discuss.

View on HN

> I find this new trend of trying to predict how technology will be used - and presuming its users will either be evil or too stupid to use it responsibly incredibly patronizing.

People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise.

> Engineers are not philosophers and should not be placed in this role.

This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher.

And sure, engineers may "just" apply technology by comparison, but honestly I find that a very dismissive attitude as well. They are the "praxis" to academia's "theory", meaning that what they do actually has consequences in our lives. So if anything, their works comes with even more of a moral obligation to "Do The Right Thing" than that of academics.

> People who create something, who research something, should by definition have a better grasp on their topic than everyone else. To expect that they are more knowledgeable, and therefore better equipped to see what it means, and therefore also have a civic duty to explain this to less well-informed, not not patronizing. It's acknowledging their expertise.

They are experts in how the technology works. That doesn't make them an expert in how other humans will take advantage of what they have discovered/created. I've taken the charity of assuming the scientist or engineer themselves had non-evil aims.

> This is an interview in Nature. Academics with the ambition to publish in academic journals, be they prestigious like Nature or Science, or smaller and more niche like <insert your favourite journal here>, are taking on a role of a philosopher.

Yes, but a philosopher on which topic? A doctorate from an engineering background is not equally as educated as someone with degrees in history or other studies of human nature in regards to the societal effects of technology.

The goal of this proposal seems worthwhile but the implementation leaves a lot to be desired.

Humanity benefits when the broader impacts of technology are factored in. Who does this, with what influence, is arguably one of largest themes in human history. This particular proposal is just one of many options.

I don't find this article particularly compelling. Having a requirement for a social-impact section is a blunt instrument; it might become just a pro-forma exercise.

I think I would be more interested in encouraging (not requiring) journals with strong ethical foundations let authors opt-in to discuss social impacts. If they do, I would expect these social impacts to be vetted by social scientists, philosophers, ethicists, and so on -- areas where computer scientists often have large blind spots.

Doctorate of philosophy.

The Ph in PhD actually means something. It's not some kitschy decoration.

The Ph in PhD refers to natural philosophy, which diverged from moral philosophy early on.

How is natural philosophy different from moral philosophy?

I'm sure that comes across as a dumb question, but, modernity - technology.

The irony of mathematics being naturally built in, reinforced into all of us, over and over. Computer science.

From wikipedia:

> From the ancient world, starting with Aristotle, to the 19th century, the term "natural philosophy" was the common term used to describe the practice of studying nature. It was in the 19th century that the concept of "science" received its modern shape with new titles emerging such as "biology" and "biologist", "physics" and "physicist" among other technical fields and titles; institutions and communities were founded, and unprecedented applications to and interactions with other aspects of society and culture occurred.[1] Isaac Newton's book Philosophiae Naturalis Principia Mathematica (1687), whose title translates to "Mathematical Principles of Natural Philosophy", reflects the then-current use of the words "natural philosophy", akin to "systematic study of nature". Even in the 19th century, a treatise by Lord Kelvin and Peter Guthrie Tait, which helped define much of modern physics, was titled Treatise on Natural Philosophy (1867).

> "systematic study of nature"

And what do we do? What is nature to study? What is life to study?

Our own puzzled face, staring back at us.

> Doctorate of philosophy.

> The Ph in PhD actually means something. It's not some kitschy decoration.

In Germany, the doctor's degrees are quite different. Typical ones for the HN audience are (for a long list cf. https://de.wikipedia.org/w/index.php?title=Liste_akademische... ):

- "Dr.-Ing." for engineering (typically including Computer Science)

- "Dr. rer. nat." (doctor rerum naturalium) for natural sciences (typically also including mathematics). Even though the university of Frankfurt am Main issues the "Dr. phil. nat." (doctor philosophiae naturalis) title instead, the "nat." ending for the latter one is very important to distinguish it from the "humanities doctor's degree" "Dr. phil." (doctor philosophiae)

TLDR: In Germany, the names of the doctor's degrees make it very clear that they have nothing to do with philosophy.

I will not be convinced into believing a PhD represents less than what it does. They are absolutely fundamental to the retention, collection, understanding, and dispersion of knowledge. Otherwise intellectual understanding goes in circles. No progress.

PhD represents what it does, it's just something else than you seem to think it is. PhD is awarded for being a specialist in a very specific topic (+ perseverance in jumping through institutional hoops), not for being a specialist and having an extra understanding of sociology, or philosophy in the modern sense.

That's absolutely not what I was taught it meant, or learned, or came to discover.

It literally looks like anyone can change knowledge, override it, mutate it, destroy it, falsify it.

I don't know why most people get PhDs these days, you are correct. But I know what it used to mean, and I hope it means that one day again, soon.

In the real world you have to jump through hoops no matter what you do. Ignorance to them is stupid.

We live in society, that doesn't make everyone a sociologist. It just means everyone who lives in society should at least devote some portion of their thinking to the contemplation of society, including institutions of education. People in academia are not omnipotently protected from the forces of society. If society doesn't value these institutions, they go away.

The Ph in PhD is important. It really is, and it's idiotic to let that distinction be viewed as something that's just, obsessed with a pedantic, tiny niche of no real consequence.

I've considered myself an autodidact for much of my life. I've also had a great education (MSc, only worked on PhD for one year and left school due to health complications). Comparing these two worlds is hard, but teaching yourself when you can be taught by others is the hard, long, and painful way to learn stuff. And you never know what you know when you only have your own knowledge to trust, checking against everything else, with no real stable, structured, foundational, 'stand on the shoulders of giants' mechanic for the discovery, pursuit, and application of truth / knowledge.

You can have some idea that the individual mind independently can be related to the entire body of academia like it's some metaphor for their functioning as an organizational entity, a super colony of minds that function like one - tested against the rest of the world - but that's oversimplified. Our own puzzled face, staring back at us. We are all part of one big system, but some stuff, distinction really does matter. I don't talk to many people who have PhDs that I know of in real life, just one. He's the only person who seems to really understand the absolute delicate fragility of, and the consequential burden - that comes with the discovery, creation, pursuit, and retention of knowledge, at least that I know of, in real life.

How does that relate to having an extra understanding of sociology, or philosophy? I honestly think my point is built into the experience I am relating, or at least I have to hope I can still convey that much, correctly.

Philosophy should be important to society, and it's a shame when it's not.

But it has zero to do with philosophy in the modern sense. Wikipedia will easily tell you that "philosophy" in this context refers to an original Greek root and an archaic way of categorizing fields of knowledge.

https://en.wikipedia.org/wiki/Doctor_of_Philosophy

Since when did Wikipedia become the authority on what a PhD represents?

That's a little freaking ironic if you don't mind me saying. A PhD represents it's own authority by virtue of being awarded by the tiny fraction of people in the world who actually understand the problem space you might be studying enough to say "Research accepted. We trust you can take our place as educators and researchers in the future when we are dead."

Also, there's a good reason the Greeks had that archaic way of categorizing knowledge. Hyper-connectivity of all knowledge available at once is a total clusterfuck to deal with from an individual mind perspective. Just because it's old doesn't make it wrong. History repeats itself, etc.

> To expect that they are more knowledgeable, and therefore better equipped to see what it means...

At least in computing, this seems to have been largely false. Whatever my respect for the expertise of creators, their track record on predicting the future uses of their developments has been genuinely bad. I don't think that's an indictment of their skills, just an acknowledgement of how complex and rapidly-changing systems really work.

Understanding the future impact of ARPANET was impossible without also anticipating the rise of home computers, but the creators of ARPANET had no special expertise on that point. Computability experts and chip designers got predictions about topics from weather modeling to AI Go play totally wrong because everything from GPU computation to neural network advances have rendered their accurate domain knowledge misleading. And the track records of AI experts predicting the real-world role of AI is so infamous that I don't think I need to elaborate.

Honestly, I expect that this would be computer science's Modernist moment. Corbusier and friends were genuinely excellent architects, but the move from "designing on sound principles" to "trusting that we know all the future uses of buildings and designing for those" was disastrous. The result is a legacy of inflexible building design, hostile urban planning, and government-embraced block housing that caused real harm to the people it was inflicted on.

"A better grasp on their topic than everyone else" is presumably true, but it's not enough. Complex systems are often beyond what any one person can predict, and that only becomes more true when they're also rapidly changing from many directions. I don't think most creators in computer science can make predictions good enough to act on until their creations have been exposed to the world for a while.

> Honestly, I expect that this would be computer science's Modernist moment. Corbusier and friends were genuinely excellent architects, but the move from "designing on sound principles" to "trusting that we know all the future uses of buildings and designing for those" was disastrous.

First of all, I love me a good Le Corbusier bash, and I totally agree with you here. I also get where you're coming from.

However, what the article asks for is the equivalent to demanding that Le Corbusier would have to ask at every step "how could my design, supposedly built on sound principles, have negative consequences?" which isn't something I recall being a thing he used to do.

Of course it won't be enough, but one could argue that if the experts don't even bother to try, things will be even worse.

> demanding that Le Corbusier would have to ask at every step "how could my design, supposedly built on sound principles, have negative consequences?"

This is a good point, and I'm very much in favor of that. The man himself seems to have been much more fond of questions like "how can I force people to accept a version of this design that uses round numbers for elegance?"

I think the proposal made me jumpy because I have so many bad associations with asking technologists to explain the consequences of their work to others. It calls to mind things like The New Digital Age, which seemed totally devoid of introspection (and frankly stupid or even malicious in a lot of places) but was a huge hit with people making important choices about investment and regulation.

I'm still very hesitant about things like the mention of public funding boards using these assessments to shape their decisions, but I appreciate your point about preemptively looking for possible negative outcomes. Something like a medical attitude that considers failures and negative events separately could potentially make a lot of sense. We live in a world where the UK police are using facial-recognition software which provides 95% false positives, and it would be nice to at least have an academic framework for talking about how that's not just ineffective but actively bad.

I understand what you mean. Keep in mind this is intended for the context of peer review; an ultimately transparent discussion, but not directly answering to the public. Ideally it would become another form of constructive criticism among scientists.

that would be great if our engineering education system placed any real value on humanities or social sciences. They don’t. Most of the engineers I know aren’t even remotely equipped to work out how their products might impact society, and to be honest even fewer actually care enough to change their behavior or take risky positions in front of their bosses.

>Engineers are not philosophers

Hold your horses, some of us engineers have philosophy degrees!

In all seriousness, engineers are more than capable of considering the impact of their creations. But can you predict future uses for technology with accuracy? Of course not.

I'm not sure how relevant it is for most work. People will always find ways to use technology to the detriment of humanity, but the overall effect of technology is beneficial. The internet is obviously for the benefit of humanity, yet there are plenty of people using it for evil.

If a person says "Engineers are not philosophers", the common interpretation is not "All engineers are not philosophers", it's "Nearly all engineers are not philosophers".

The small set of engineers with philosophy degrees does not negate the parent's statement.

Did you just pick a formal logic debate with a philosopher? I disagree with your interpretation of the language used. The comment was authoritative in tone generally, and this line said (in my estimation) that the argument was "if someone is an engineer, they are not a philosopher". Even worse, there's a suggestion that this philosophical work shouldn't burden those "least qualified to perform it". Leaving the philosophy to "better men" is how we justify evils committed by underlings and evokes thoughts of the Nuremberg trials. Every person should come to terms with the moral implications of their actions.

In any case, one doesn't need a philosophy degree to do philosophy, and many of the great engineers and scientists of history had a great deal to say with regard to the moral implications of their work.

Even today, engineers at many of the big tech companies are objecting to some work their companies are doing. Many others choose not to work for companies whose mission they disagree with.

No formal logic debate here; I was explaining colloquial use.

Then don't call yourself an engineer. Engineers build bridges, airplanes and when those fail, engineers are investigated for culpability. If the failure is due to a design flaw or cut corners, engineers are held responsible. I say this as a computer scientist with an undergrad in eng and a PE.

Those are first order effects connected directly to the immediate aim of allowing transport over an obstacle. A civil engineer does not consider "who" will cross or what their aims will be. For example a bridge may allow an enemy army to move their tanks faster during wartime.

Similarly tech shouldn't have to consider whether someone with unrelated goals will use their tech for unwanted aims.

If you know your software will increase emissions to illegal levels, and still write it, you are culpable.

If you write software for autonomous cars that is not foolproof at detecting pedestrians, you are culpable. In aviation, catastrophic failure means at least one death and the system should have an overall probability of failure at 10^-9. Critical software is frequently written at one or two orders below. That means 95%, 99%, 99.9%, are not enough.

What are the limits of culpability and what are you responsible for? The article is pushing that definition beyond the first order effects of safety. And the proposed solution is that gate keeping is done by the press and the public. How well do you think that will go?

If you write software that replaces someone's job are you "culpable"? If you create a phone and people are killed when someone texts and drives are you "culpable"? If you create a better facial recognition program as part of building a secure access systems and someone uses that system to discriminate are you "culpable"?

I find this unending process of creating "victims" just so they can be "saved" as tiresome.

I find engineers and entrepreneurs who shirk their responsibilities as moral agents tiresome.

If a thief comes and steals money under the mattress meant to pay for your kid's college tuition, and your kid is dropped because she can no longer pay, your kid's loss is a second order effect of the thief's theft, but I still hold the thief responsible*

* this is a hypothetical story; please don't save that kind of money under your mattress.

Your argument has some basis in law and I do find it compelling. The nuance to my argument is that the original creator did not have nefarious aims. I believe that does make a difference.

I think that's fair. I like to consider three factors when considering an original creator:

1. Did they have nefarious aims?

2. Did they reasonably attempt to consider and solve issues surrounding first and second-order effects of their creation?

3. How did they react to previously unknown first and second-order harms when they found out about them?

The first two, I tend to give creators the benefit of the doubt about. But I've become quite dismayed by the responses of creators to the harms of their creations.

I assume the common response to #3 is that the benefits outweigh the costs. Do you not consider that a compelling argument?

Benefits to whom? The engineer and entrepreneur? Or the public at large?

Benefits to management when the public actually believes the culpable will be punished when shit hits the fan.

Tell me, in the VW emissions scandal, did that happen ? No.

Germany literally changed the laws retroactively to avoid holding their own companies to account. This is technically illegal, but the judicial system REFUSED TO LOOK AT IT.

Given that both management of these firms, and government wants to commit fraud when culpability is assigned ... what point is there discussing the rules ? They won't be applied when it matters.

But of course there are large advantages to people believing they will be applied. Like there are advantages to people not responding to having their wallets taken from their pockets, their children kidnapped for sale into abuse, their houses robbed empty ...

This is the US government in action. Other governments, including European, are worse, not better. These are the people that make, and enforce the rules. Why are we discussing whether the right things will be happening ?

https://www.youtube.com/watch?v=eqBAOX6Qegk

And if her college boyfriend becomes depressed after she leaves and loses it, gets drunk, and hits someone crossing the road while driving, I still hold the thief to have some moral culpability. Moral culpability is shared and distributed across moral agents in time and space, as some function of causal distance and intent.

It is the second word, intent, that is the hang up. If a person hurts someone but it was not their intent to do so, that person is not typically to be held morally responsible. So, at least in the western concept of moral culpability, intent matters. But... people also have a responsibility, a responsibility to try to anticipate the consequences of their actions. One cannot shield oneself from culpability by intentionally remaining ignorant. I think that engineers avoid thinking about these things, not out of humility, but because its in their interest to do so.

> Moral culpability is shared and distributed across moral agents in time and space, as some function of causal distance and intent.

Absolutely agree.

> I think that engineers avoid thinking about these things, not out of humility, but because its in their interest to do so.

This is perhaps the reason we differ. My experience has been that the vast majority of technologists want to make the world a better place.

I believe you see the industry is filled more with people like Wernher von Braun who want to build something cool regardless of social cost. At the dawn of the social media revolution the dominate theory in tech circles was that information shall set you free. The people making it did not come into the industry for status, being a nerd was still a social negative. Today with the ever rising salaries things are changing.

Perhaps you are right that as the industry matures the sphere of liability should increase. I would still argue that the suggestions in the original article swing that much too far. We shouldn't prevent scientific discoveries merely because engineers may build something bad with the information. On the balance scientific discovery has improved the lives of everyone by many orders of magnitude - even if its distributed unevenly.

That's what standards are for. The threshold is determined by harm. Software practices are still in their infancy, which is is why it is hard to take them seriously as mature disciplines. Now that software is having a direct impact on people's lives on a large scale, it's time to set some standards, starting with autonomous driving and mass surveillance. Then have people trained to understand those standards and adhere to them as best practices.

If you do it knowingly yes...you have the ability to say no. Do you think you are not a victim, or are you just complacent enough that as long as you have X you are fine and I can going knowing I was a good person. You dont have to create victims you do have an obligation to prevent them.

> If you know your software will increase emissions to illegal levels, and still write it, you are culpable.

Again a first order effect

> If you write software for autonomous cars that is not foolproof at detecting pedestrians, you are culpable.

Again a first order effect connected directly with the immediate goal.

However if you write communications software and people use it to say bad things, those are second order effects. You provided communication software that works as advertised and has beneficial purposes. Others misused it.

Is the guy who invented valves responsible for the Holocaust?

Not trying to be inflammatory. This is specifically trying to find if you think there's a point where culpability disappears. I would appreciate a yes/no answer. There's no nuance to the question. I'm not talking about the specific valves, just the notion of a valve.

Do you understand the difference between a first order effect and a second order effect?

I don't (btw, I'm a software engineer and scientist with extensive training). So I looked it up.

Did you mean first order or second order kind of like the way physics people mean it (direct linear relationship with strong effect on the unknown variable). Or the way that MBAs mean it (they have their own definition).

Or did you mean structural engineering, same as physics: https://www.quora.com/In-structural-engineering-what-are-fir...

I think what you probably mean is a colloquial interpretation, where first order means something is directly causal in an obvious to see way, where second order is more of an indirect effect that occurs through complex dynamics.

Either way, I don't think those terms are well defined, and they're kind of complex enough that using them to argue for morality, and calling people out for not knowing your definitions, makes you look caddish.

I agree.

I doubt that they could mean causally indirect. I think it must mean something like: not used in accordance to their the engineer's intent, especially when that intent covers a a general affordance (e.g. affords communication over long distances), but is being judged for a more specific affordance (affords communication over long distances to order a terror attack). I dunno. I'd have to be a philosopher to work it out.

I think that's spot on. When people say something is a first/ second/nth, order effect they are really just arbitrarily drawing discrete lines in a continuous chain of causality.

The problem of where to draw those lines is so difficult that legal systems resort to constructing tests based on whether a "reasonable person" could have foreseen the specific consequence of an act.

Those are also, as the OP has repeatedly addressed: first order effects.

Yeah but that’s a different argument to what you’re replying to.

If that civil engineer sold or allowed (for any reason from negligence, insufficient security protocols, malice, ignorance etc) blueprints for his design to end up in enemy hands, he most certainly would be held liable.

The thrust of that point is the very idea that software engineers on projects involving user data, regardless of how and what they think they are building, are not simply building “bridges”. They are building systems that have the potential to be misused as mass-surveillance apparatuses. And they need to build their systems accordingly, with appropriate data-security and with acknowledgement that their work could leave them personally liable to action should their work be found to be used as a threat-vector towards their users.

The current attitude of “well people know what they’re signing up for and since they signed the User Agreement, we don’t have any liability here” towards users in the industry is sick.

I literally signed up for HN, something I've been uninterested in doing, just because of this comment.

Saying that engineers need to make responsible decisions in their software design is one thing. But you imply that misuse is a liability for the designer? So if someone takes an engineer's code and modifies it for malicious intent the engineer is liable? That's absurd.

That's like saying the person who designed the airplanes used in 9/11 are partially liable for 9/11.

Absolutely, if that data is stolen due to the designers’ neglect in the security aspects of the systems they signed off as “adequately designed”.

Yup! It just concerns me since this topic has a lot of great discussion but many people are missing the distinction between different types of effects. Sorry if I misunderstood what you were saying! For others who are confused what I mean...

Consider:

The person who makes a car that someone uses to kill someone by running a bystander over. The engineer isn't responsible.

The person who messed up the Prius with the issues where the gas pedal or whatever didn't work. The engineer IS responsible.

The first example is analogous to a software engineer writing image recognition software for a safety app. Then someone maliciously taking the software and using it to discriminate via misuse. The engineer is NOT responsible. If someone takes your software (could even be open source) and applies it or modifies it in a malicious way they are responsible not the engineer.

The second is example is where if an engineer makes a widget app and has a vulnerability but signs off on it, and then widget purchaser credit card info is stolen. The engineer is responsible for negligence and should be liable for the misuse.

No, yeah sorry, I’m obviously being super brief here to just get the point across without writing a thesis on my phone :)

Although, predictably enough, I followed up with more thesis length posts to clarify my point! Namedropping einstein too, who do I think I’m impressing? (But Einstein is relevant to this debate, although he made the crucial mistake of only informing the government and not informing the public. Obviously this was well-intentioned on his part and not malicious at all but it’s a lesson that every developer of new innovations needs to take onboard.)

This isn’t about specifically demonizing “engineers” or innovation. Or even to do with liability where “that engineer developed a car that murdered someone”. Liability laws are robust enough to figure out whether the gas pedal was functioning correctly, the driver is at fault, or if not, the engineer (or rather, the company he works for) is at fault.

This debate is about broader strokes: should developers of new technologies be ethically bound to inform both politicians & the public about their innovations, specifically so that, after a public debate on the potential negative consequences of that technology, the public can then demand their politicians to produce robust laws protecting the public from misuse of those technologies. (I say yes, I know it’s tough because I love just developing cool new shit too, but we developers have got to take a step back and realize the future implications of what we make. Data gathering, manipulation and retention is a ticking time bomb. And who knows what’s next.)

Another example is facial recognition, it’s depressingly funny to see experts who know so much about the technology proclaim it to be no big deal without understanding the ramifications of living in a society where your every movement is tracked and potentially monitored. That is a literal police state, and the US constitution has amendments made hundreds of years ago specifically to outlaw such actions to protect the public at large, both from private companies tracking their movements and to protect the public from the government itself. To have all of that progress in the name of “facial recognition will be awesome because you can login to your phone quicker” is not a good move because once that technology is in the wild, until laws catch up to robustly protect the public, who knows what will be done to the public at large with that technology. Ditto for “who cares about data retention and website tracking, it makes marketing so much easier!” and “don’t worry about your DNA being stored and sold by ancestry sites indefinitely because it’s so cool to know that you’ve got a 7th cousin living on a different continent”.

> engineers are investigated for culpability

Funny how you believe that. Remember the Miami bridge collapse ?

Workers on the ground misplaced one of the supports, and then didn't stop work when loud noises started coming out of the concrete. This was not documented wrong on the design, it was those same companies first mismeasuring the location of the supports, then simply moving them around when they didn't fit.

Who's culpable ? The designers, of course.

Or take Dieselgate. European, German and French car manufacturers systematically and purposefully installed software in the cars to commit fraud on the testers (the government testers knew that this was happening).

Who's culpable ? A single engineer, of course.

Not the regulators that knew what was happening.

Not the European investment bank, who had seen reports on this.

Not Volkswagen management, that had built several facilities explicitly for committing this fraud.

Let's stop pretending the problem with engineering will be solved by assigning blame to engineers. It won't. It can't.

Large scale fraud, committed by management and government is the problem. It cannot be fixed with engineering.

What!? Professional engineers do consider the greater good of society in their code of ethics. Introductory courses in ethics are a mandatory part of the curriculum where I’m from.

And yet that is not even close to the hand-wringing being discussed in the article:

"In my first ever AI class, we learned about how a system had been developed to automate something that had previously been a person’s job. Everyone said, “Isn’t this amazing?” — but I was concerned about who did these jobs. It stuck with me that no one else’s ears perked up at the significant downside to this very cool invention."

"Then, the gatekeepers are the press, and it’s up to them to ask what the negative impacts of the technology are."

Building an entire society of victims and allowing the hysterical press, who are in the business of feeding a hysterical public sounds like a formula for paralysis.

I'm appalled at the lack of ethics in civil and mechanical engineering!

Automotive engineers willfully destroyed opportunity for stablehands, whip manufacturers, and train conductors out of hundreds of other occupations annihilated.

Civil engineers with their oriented strand board trusses, reinforced concrete, and dimensional lumber stole the livelihoods of masons, carpenters, foresters and so many more.

Engineers have centuries of unethical practice to atone for. They need to stop their work because everything that they do is harmful and unethical!

I guess a little humility is warranted. But I think your attitude is really about letting engineers shirk their responsibility as moral agents. You know, sliiiiide by, as they've always done. Take responsibility.

> But I think your attitude is really about letting engineers shirk their responsibility as moral agents.

I absolutely disagree. That is why I specifically mentioned culpability for first order effects. If you build a gun for someone you know to be violent you have culpability for its use. What I disagree with is the broadening of this "liability" to include second order effects. That we are now supposed to predict chaotic systems and know in advance all perverse outcomes - and that we should have moral liability should we fail in this task.

we have moral liability for second order effects (whatever those are)

An abuser drives over a bridge to reach his victim's house, where he commits an act of domestic violence.

You've committed to the position that the engineer who built the bridge is morally culpable for that domestic violence.

Perhaps, but is Anubis' scales, the good is weighed against the evil that a man has done. Cannot the engineer be culpable for facilitating a crime, and yet be also culpable for improving the lives of many thousands of others, and in balance the good outweighs the bad? Culpability is not an absolute. The engineer may be culpable, but how culpable? Relative to the culpability of the one who held the knife? Relative to the man's parents who suspected his intent but failed to stop him? Relative to a society which has decided that it needn't guarantee access to mental health care that the man required? Culpability is distributed, and it isn't all or nothing.

You stated that we have moral culpability for second-order effects, whatever those are. If a person engineers a bridge that works as designed, what would the second-order effects be?

"whatever those are" was a poor attempt to indicate that I wasn't sure what people mean by second order effect. was it that something that was not intended, or is it that it happens further in the causal chain, or is it something about particular affordances and general ones (a bridge affords crossing. a bridge that affords crossing both affords crossing a bridge to buy your groceries and crossing a birdge to commit a crime.) Or is it just things that are hard to predict might occur? Or things that have an intervening autonomous moral agent between the engineer and the victim? its not clear to me.

A man-made dam might increase a city's availabe water supply and allow power generation, but it also can increase CO2 emissions and do harm to salmon reproduction. Increased CO2 emissions and detriments to salmon reproductions might contribute to global warming in the first case and an increased price on fresh fish and a declining seal population in the second. Increased price of fresh fish mean less availability to poor consumers and the loss of what might be an otherwise valuable part of diet. Global warming means climage change, sea level rise, an expected increase in unrest, and large scale ecological instability. All those things mean that my daughters may not enjoy the relatively secure life that my generation has led...

On what basis?

on what basis are we culpable for anything? if you can answer that I question I'll try to answer yours. not trying to be flippant (we've had a good discussion so far), but I don't really have an answer, except to say that I don't know on what basis we might distinguish the two cases. if you can answer me what makes someone morally culpable of anything, maybe I'd have a clue.

The basis for my arguments are from English law. Who is guilty of what has been fine tuned over centuries. It's not enough that something bad happened. There must be either intent to harm or gross negligence.

The modern trend both in law and society of statutory offenses and ex post facto "something bad happened so let's see what we can charge him with" is something I find deeply disturbing. I find the original articles proposals as yet another step upon that road.

Our forefathers accepted the universe was unpredictable. We seem to feel we have enough grasp now that if bad things happen someone must be to blame.

> Engineers are not philosophers and should not be placed in this role. We do not have the tools to do it. As with any human we have a duty to consider first order effects - but even the most astute philosopher will have trouble anticipating second order effects. This burden should not be placed on those least qualified to perform it.

Here are two rebuttals to the above statement.

1. At the level of projects it's asking engineers to think a bit more broadly about solutions. For instance, when designing a user comment service for social media you can trade-off between designs that encourage informed debate vs. ranting flame wars. In many cases both designs have similar implementation costs and meet business requirements, so why not pick the one that is more socially responsible? As these choices become better understood they should just be part of user experience design.

2.) To develop the trade-offs we should address more research to social consequences of technology and take the results seriously. Great engineers are philosophers or at least more broad-minded about the interactions between technology and society than some of the current leaders in Silicon Valley. Andy Grove, David Hewlett, Bill Packard, and Bill Gates were/are not just problem solvers but examples of engineers who have thought about how to benefit society as a whole.

> Engineers are not philosophers and should not be placed in this role.

Which is how we wind up with companies hell-bent on vacuuming up as much data as possible and selling it to anyone who asks. Perhaps if some software developers (or, at this point, some legislators) stopped to think about the ethical considerations of the tech industry’s actions, we all wouldn’t have to have conversations about how other countries can buy elections on the same platform where you share pictures of your food.

You're kidding yourself if you think that electioneering and lobbying are a new thing. Look at Saudi role in promoting Wahhabism, religious fundimentalists worsening AIDs epidemics and promoting homophobia by pushing bullshit on the third world for one alone let alone actual CIA or KGB influence. Before people were falling for literal fake news they were falling for Satanic Ritual Panic because the truth wasn't what they were concerned about but how it made them feel.

Tech was never the issue here - people were. The drive to scapegoat the upstarts should be regarded with great suspicion. While there may be questionable ethics involved with companies it looks to be a classic attempt to scapegoat the troublesome for matters that are peripheral at best and any action they could have taken would have dissatisfied people like how both the left and right are convinced Facebook is biased against them because they don't reflect the world as they wish to see it.

I have a slightly different take: engineers shouldn't be blamed for negative outcomes that propagate down from the political and cultural levels.

For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights. Thing is, I imagine that if, say, ML researchers had gone on strike over privacy issues somehow, society would have essentially told them to shut up and get back to work. Sure, the same larger society now wants to blame ML for the constriction of privacy and for extensive state and private surveillance networks, but it was that very society which cheered the construction of these exact surveillance networks "becuz terrorizm" and for the sake of "free speech" (for billionaire multinational corporations) and for other awful reasons.

The people who were skeptical or critical over this whole social trend knew damn well it was all a bad idea, and we said so. Society deserves to get what it voted for, with both its ballot box and its dollars, and on some level, it deserves to get it good and hard without scapegoating the hired help, when it was them in the first place who told us skeptical, critical engineers and scientists to shut up and produce what society wanted from us.

I agree but have a bit different take...

"For instance, machine learning is now widely used for surveillance and advertising in ways that violate basic presumptions about privacy rights."

Those same mathematical techniques are also being used to more accurately detect cancer in MRI scans. Saying everyone needs to be held responsible for second order effects is unrealistic and ultimately damaging to society. Blaming the toolmaker for things they cannot control will only cause no tools to be made.

To me it points to a larger issue in our society that we are on a trend of completely removing context. Context no longer matters, what matters are the effects, regardless of whether you can reasonably be held accountable for them or not. It is thought-crime induced by those who in the same breath rail against that totalitarian behavior and then skip to implementing a version of it themselves.

A number of people on this thread immediately jump to the argument on surveillance and data privacy which is a useful discussion but tangential to what is being proposed in the original article. The article is moral hand-wringing with zero clarity or practicality. And the judge of who is culpable or not is left to a hysterical press, driven by a hysterical and sometimes uninformed public.

Look at what this behavior has done to political (and personal) discourse. Now imagine it being applied to science. No thank you.

>Those same mathematical techniques are also being used to more accurately detect cancer in MRI scans. Saying everyone needs to be held responsible for second order effects is unrealistic and ultimately damaging to society. Blaming the toolmaker for things they cannot control will only cause no tools to be made.

As a matter of fact, my work covers fMRI scans of brains for a neuroscience lab, so I'm entirely with you here, but again, with a small caveat:

If society wanted the "toolmakers" (us) to take responsibility for which tools we make and how others use those tools, they would have to give us the actual power to decide what we work on and for whom. Instead the rule has usually been, "if you have ethical qualms about this, we'll find someone who doesn't". All exit and no voice means a society where nobody takes responsibility because everyone self-selects into positions where they can agree with what they're doing, and if broad masses and the hysterical press want to excoriate those doings while also materially demanding they continue, it gets to do so.

It's like when people say that engineers should refuse to work on weapons that would violate the Geneva Conventions -- but the same people then vote for a government that moves funding from the NSF to the Defense Department!

There are more knowledge creators/workers than ever before. I don't think that expecting more information from any given study (especially if it is applied research) is detrimental to society or the study itself.

It's about raising the standards because we can. Standard publication sections: intro, material and methods, results, discussion, conclusion. It feels like you can add a "speculative impact" section somewhere between discussion and conclusion.

Computer Science is a new field and individual developers can have reaching effects on society, direct or indirect, that extend further than any other engineering discipline. My belief is that it 100% is on the responsibility of researchers and developers to think of the ethical and social implications of their work. As an example, a podcast I listened to recently was interviewing CS researchers working on "deep fake" videos. When asked if they thought their work could have negative societal impact, their response was along the lines of "its just really cool to make this". CS as a whole has a lot of maturing to do and I hope we can get their sooner rather than later.

That's why diverse teams are important. You're right, we can't expect a CS major to understand social impact, but if you're working on AI or weapons out some other high-risk project there should be someone in the team who does.

I have to agree. I think that calling it "patronizing" is harsh but science generally does not progress on a timeline that depends on and adapts to humans - it is the other way around. And whereas history may judge the oil industry negatively it also judges theocratic regimes negatively for trying to "mitigate" progress.

The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, [274e] “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; [275a] and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess.

Socrates, From Plato's dialogue Phaedrus 14, 274c-275b

http://www.historyofinformation.com/expanded.php?id=3894

Totally agree. And this is not new at all.

All you have to do is read the history of engineers and scientists who worked on nukes or rocketry. The ones that tried failed miserably. Understanding why is crucial in understanding the limitations of domain experts.

There is a big difference between computer science and computer engineering.

Science is about understanding. Engineering is about using that understanding for building.

Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world. Science and Engineering are inextricably linked in this discussion.

> Science and Engineering are inextricably linked in this discussion.

Of course they are, but this article is very specifically about the role of the scientist. Not the engineer.

Will negative disclosure cause problems for engineers? I don't know. What I do know, is it isn't fair to place that burden of knowledge entirely on scientists while tying their hands behind their back with respect to open discussion in a professional setting. Academic publishing is intended to be open without fear of professional and personal repercussion for increasing factual awareness. Get rid of that and, well, sit back, relax, and I hope we all can enjoy the apocalypse of knowledge, because that's where stuff is headed without the ability to professionally discuss existing problems.

> Nobody would be discussing the moral implications of scientific discoveries if it didn't ultimately lead to machines that cause effect in the real world.

That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible.

> That's just factually wrong. Mathematics. Philosophers. Logicians. All the living and dead theorists who built the abstract foundations essential for the functioning of our machines. People who deal entirely in the world of the abstract. You don't develop a passion for abstraction because you want something out of reality. You develop a passion for abstraction because reality is sometimes terrible.

We don't have moral discussions about the discovery of a new law of physics or a mathematical lemma. This only becomes an issue when we can see how it can be applied to some aim. Thus if there were no engineers nobody would consider the scientists moral obligations with the exception of their experiments themselves causing harm.

The argument to limiting science is that engineers might take it and create some terrible machine with it. Perhaps the most notorious example being a nuclear bomb.

I believe we agree on the role of science, I don't think we agree on what their culpability should be.

> We don't have moral discussions about the discovery of a new law of physics or a mathematical lemma.

Abstraction affects mental functionality. It forms expectations by designing a language in which the present can be expected to be predicted within some objective, measurable tolerance. The problem, always has been, and always will be - is that people can't be measured and tested like machines.

> This only becomes an issue when we can see how it can be applied to some aim.

So yes. That's correct, that's the terrible thing that can be observed and measured when it happens. Prevention is important.

> Thus if there were no engineers nobody would consider the scientists moral obligations with the exception of their experiments themselves causing harm.

Sorry, no, that again, ridiculous. You think that way. Not everyone thinks the same. Intent. Experimentation is intended to discover in a controlled environment while giving heavy weight to moral implications. People choose to opt in to scientific studies. People trust that process when they make that decision.

> The argument to limiting science is that engineers might take it and create some terrible machine with it. Perhaps the most notorious example being a nuclear bomb.

Yes, and what makes us think we are so evolved and so advanced to believe we can't accidentally create something similar with present technology - lacking awareness to it's creation? Evolution won't protect us against our own idiocy.

Reasoning about computation affects cognition, which has real world consequences that extend to everything else that is not reasoning about computation. That's just all there is to it.

> I believe we agree on the role of science, I don't think we agree on what their culpability should be.

They should be able to talk about personal, first hand experiences openly without being professionally attacked, first. Second, they should be able to talk about existing problems without fear of whose bottom line it might affect. There should not be blame in systems where everything is hard as heck to understand. Obligation to the truth - yes. Blame for the belief that there was the intent to create a problem by virtue of uncovering one? No. That has shortsightedness written all over it.

I would like to engage more substantively with you, but I don't think I understand where you stand on this issue. Do you agree or disagree with the suggestions in the original article?

> Brent Hecht has a controversial proposal: the computer-science community should change its peer-review process to ensure that researchers disclose any possible negative societal consequences of their work in papers, or risk rejection.

Agreed.

> The idea is not to try to predict the future, but, on the basis of the literature, to identify the expected side effects or unintended uses of this technology.

Agreed.

> No, we’re not saying they should reject a paper with extensive negative impacts — just that all negative impacts should be disclosed.

Agreed.

> But we’re moving towards a more iterative, dialogue-based process of review, and reviewers would need to cite rigorous reasons for their concerns, so I don’t think that should be much of a worry. If a few papers get rejected and resubmitted six months later and, as a result, our field has an arc of innovation towards positive impact, then I’m not too worried. Another critique was that it’s so hard to predict impacts that we shouldn’t even try. We all agree it’s hard and that we’re going to miss tonnes of them, but even if we catch just 1% or 5%, it’s worth it.

Agreed.

> We need to be saying, based on existing evidence, what is the confidence that a given innovation will have a side effect? And if it’s above a certain threshold, we need to talk about it.

Agreed.

> We believe that in most cases, no changes are necessary for any peer reviewers to adopt our recommendations — it is already in their existing mandate to ensure intellectual rigour in all parts of the paper. It’s just that this aspect of the mandate dramatically underused.

Agreed.

> Then, the gatekeepers are the press, and it’s up to them to ask what the negative impacts of the technology are.

Agreed.

> Disclosing negative impacts is not just an end in itself, but a public statement of new problems that need to be solved. We need to bend the incentives in computer science towards making the net impact of innovations positive. When we retire will we tell our grandchildren, like those in the oil and gas industry: “We were just developing products and doing what we were told”? Or can we be the generation that finally took the reins on computing innovation and guided it towards positive impact?

Agreed.

I haven't read this article (https://acm-fca.org/2018/03/29/negativeimpacts/) in entirety but I agree with the bolded points. Only viewed bolded points up to the examples.

Edit: I want to say thank you for your respectful tone in spite of my slightly aggressive(?) one. I'm grateful to people like you because it reminds me, it matters a lot.

> The idea is not to try to predict the future, but, on the basis of the literature, to identify the expected side effects or unintended uses of this technology.

What will you do with this information?

> No, we’re not saying they should reject a paper with extensive negative impacts — just that all negative impacts should be disclosed.

So at best its useless, and at worst it serves to fuel the imaginations of those who would use it for evil.

> our field has an arc of innovation towards positive impact

How would this change the course of innovation? The only way I can see is if its used to censor.

> We need to be saying, based on existing evidence, what is the confidence that a given innovation will have a side effect? And if it’s above a certain threshold, we need to talk about it.

Why is a specialist in a specific technology qualified to determine negative societal outcomes of it? Human societies are extremely complex systems and even those specialized in it have poor predictive track records.

> Then, the gatekeepers are the press, and it’s up to them to ask what the negative impacts of the technology are.

Why would the press have this power? They've never had such substantial control before. Governments have been able to suppress independent nuclear research in the most extreme case but the laws surrounding that are not exactly proven. If we barely allow our government this power why should we give it wholesale to an unelected press?

First, this is hypothetical.

> What will you do with this information?

If I was in the position of a researcher, I would talk to people it affects directly or try to communicate my awareness to people who are in a similar interdisciplinary dilemma who can then communicate their awareness to people who are 'on the ground' such that the problems both researchers have awareness of can be identified and then resolved in real life. The point is progress, one direction that moves forwards, not backwards or cycles. Obviously this can be a state of mind one can choose to see or ignore - see the cycles only or notice the differences with positive improvement universally in mind. But I'm one person, stuff like this will always be is a group effort.

> So at best its useless, and at worst it serves to fuel the imaginations of those who would use it for evil.

Group effort, squish evil. Evil is not something I know how to identify though, that's always been the dilemma. The real evil seems to be the thing that actually completely extinguishes your existence entirely, death(?).

> How would this change the course of innovation? The only way I can see is if its used to censor.

I don't think the intent is to censor, I think the intent is to alleviate existing censoring of problems people are aware of but have no way of speaking about.

> Why is a specialist in a specific technology qualified to determine negative societal outcomes of it? Human societies are extremely complex systems and even those specialized in it have poor predictive track records.

Maybe they have some first hand experience that they feel they would be embarrassed, ashamed, or rejected from disclosing that information because the association of being personally identified with a given experience is just something not accepted, presently, professionally. Relegated to therapy, even though, there might be actual problems people have some awareness of. Fear of being implicated as being the cause rather than the solution.

> Why would the press have this power? They've never had such substantial control before. Governments have been able to suppress independent nuclear research in the most extreme case but the laws surrounding that are not exactly proven. If we barely allow our government this power why should we give it wholesale to an unelected press?

It's not power. It's trust. It's saying, researcher talking to reporter, 'hey if we make mistakes, point them out, because we won't always be able to see every error we make, especially when we must work in a system that requires minimal error to be seen as a function of our own approval mechanic.'. Errors in knowledge systems that function off of principles that are resonant of their own structure - that leads to destabilization of structure. There has to be some tolerance for making error, that's give and take, you see me make errors, I don't see myself making them, point them out. That's the trust given to the press. Speak up.

> If we barely allow our government this power why should we give it wholesale to an unelected press?

Bodies that self regulate must regulate one another. Checks and balances.

I disagree because part of the job of engineering is stake holder analysis, where engineers consider all the groups that will be affected by a system before developing requirements of the system. This is one of the things that differentiates engineers from technicians. For CS to be more than a field of coders they should consider the larger impact of their work.

Reply on news.ycombinator.com