Hacker News Reader

Comment by godelski | Hacker News Reader

As software engineers/computer people, we have to have ethical standards. If you participate in writing code like the one in the VW case then you are not only complacent but active in fraud.

Engineers have a code of ethics. If you were to knowingly build an unsafe bridge then you would be liable for the damage and injuries it caused if/when it failed. This is true even if your boss told you to do it. You not only have a moral/ethical obligation to not build that bridge, but also a legal one.

As we write software we need to keep these things also in mind. We are writing things that affect peoples lives. That could kill people. That affect the environment. We need to hold ourselves to the same moral obligations as other engineers. We have a duty to prevent harming people when we write our software.

If you have no stand to refuse to do the work then it is your duty to report such actions. First up the chain, and if that is rejected, then to the media. It is also your duty to report it if you have refused and you see it still happening. There are whistleblower rights in place to protect you.

View on HN

Software engineers don't understand they're not professionals. I mean, they describe and think of themselves as such, but they are not legally such.

Being a "professional" legally means that you're given exclusive rights to perform some economic activity and in return for this (usually) lucrative right, you have a great number of obligations - being responsible for your work (ie, facing the possibility of never being able to perform said economic activity legally again), an enforceable code of ethics, an dispute / resolution process etc, steep fees, continuing education, an accreditation exam, etc.

Software engineers by and large do not want the obligations and restrictions of being a real profession, so no, they don't have any special ethics or rules to report or do anything in these cases beyond what a designer or secretary would do.

I've never heard of this legal definition of "professional" that you put forward. Where is this in state or federal law?

In law school, we were taught that a professional has obligations that go beyond his or her own self interest. Unlike a real estate developer or a widget manufacturer, an attorney has an obligation to do what's right for the client even if that causes the attorney to lose money or imposes a competitive advantage. Attorneys, for example, can't engage in the kinds of advertising speech that would be entirely lawful if done by a car dealership or furniture store.

This traditional definition is hundreds of years older than the bar exam or ABA code of ethics, and older than formal graduate law school (a 20th Century American invention). All of that came later, after lawyers understood themselves differently.

I would encourage people who write code -- software engineers, developers, coders, hackers, however one chooses to describe this work -- to think of themselves as professionals with obligations that go beyond their own self interest.

I would imagine OP is referring to "Professional Engineer".

https://www.nspe.org/resources/licensure/what-pe

Every single design, bridge, etc is certified by a Professional Engineer. Your ass is on the line. I don't mean just being fired from your job - but jail time.

Per the website:

"A century ago, anyone could work as an engineer without proof of competency. In order to protect the public health, safety, and welfare, the first engineering licensure law was enacted in 1907 in Wyoming. Now every state regulates the practice of engineering to ensure public safety by granting only Professional Engineers (PEs) the authority to sign and seal engineering plans and offer their services to the public. "

I suspect, and hope, software used critically have the same. Yes it's more bureaucracy and overhead...but being in software, I know any joe schmoe can walk on and develop code without oversight.

I'm not saying if you're making the next twitter, that you need a PE. Or that every developer needs to have a PE license - but like other industries there is one PE per X amount of non-PE engineers.

Due to the Threac-25 incident (https://en.wikipedia.org/wiki/Therac-25) there are standards for medical software (https://en.wikipedia.org/wiki/IEC_62304).

I'm not in medical so I don't know how widely adopted, or how strict it is adhered to. But it might be IEC 62304 needs to be branched out.

In the US at least, the regulation of software in medical devices regulates the process by which the software is developed and tested.

The qualifications of the people developing the code are somewhere down in the process-stack; there are no specific requirements [1].

In general, to follow "Current Good Manufacturing Practices" (a term of art in the FDA design controls regulation), qualifications for the people on the team have to be documented and a paper trail kept up-to-date, to be able to answer questions of the nature "was person <X> qualified to be working in this role? How do you know?" with objective evidence.

It's up to the judgment of the engineering management of the project what they state the requirements of the role to be. If you say that it takes a high-school diploma or better to be able to develop life-safety-critical medical device software, you may pass a process audit since you document and track that requirement, but if there is an "adverse incident" I would think whoever signed off on that decision might be exposed to liability.

[1] My specific knowledge about this is ~5 years out of date so this may have changed, but I doubt it

> I would imagine OP is referring to "Professional Engineer".

In part yes in part no. PE's will be held to a MUCH higher legal standard but every engineer is taught to hold themselves to a high ethical standard. That you are not supposed to do something dangerous even if you think you won't get in trouble for it. Even if it isn't directly your responsibility you are supposed to report something that looks dangerous. PE's do the final sign off and their ass is on the line much more than the engineers that actually did all the building and designing. But the code of ethics is supposed to be there all throughout the chain.

So what I'm saying is that in software we are part of that chain. And therefore the ethical obligations should still hold. They should hold at every part of that chain. At no point in the process should anyone say "well it is someone else's ethical obligation or concern". If you touch something, if you see something, then it is part of your ethical obligation. At least as an engineer, but I would argue that it is also if you write software.

PEs exist for many branches of engineering. Including software. But the PE for software engineers is apparently being discontinued due to lack of interest.

Yes, NCEES (the body that administers the Engineering licensing exams nationwide in the US) is discontinuing their Software Engineering exam next year because only ~80 people have taken it since it was created in 2013.

I think the biggest problem they had was that despite having the exam, not many states actually issue Professional Engineer licenses in the Software Engineering discipline. Because of this, I don't see why anyone (in those states) would bother taking the exam if they can't actually get a license and use the title "Professional Engineer".

I forgot licenses were by state. It's probably also the case that software engineers don't normally have to "sign off" on things for regulators in a formal way as is the case in other areas of engineering. So, other than having a title to stick on your business card, taking the PE exam doesn't buy you a whole lot whereas it's virtually required for other engineering branches.

Well, "practicing medicine without a license" is a crime. I mean, you can call yourself a "healer" or something and claim to help people in some general, non-specific way, but you can't prescribe meds, cut people up, etc etc.

So if you're a doctor you're given exclusive right to perform the economic activity of doctoring. In return you have a steep set of obligations, which include being truthful, ethical, not refusing service, etc.

What you said is a part of these "obligations", which definitely put you at an economic disadvantage, but that's ok, because you have exclusive rights to that activity. A doctor's obligations do impose costs on them, but not compared to other doctors.

The critical bit I'm trying get at is enforceability. If a doctor acts unethically, they can be challenged in front of their board by a patient and if they lose their license they can never practice medicine again. Having your livelihood at stake is a great incentive to act properly.

A software engineer such as myself, OTOH, has no little incentive beyond my own personal ethics. In many cases, being ethical will place you at an economic disadvantage compared to other software engineers. Actually, the whole notion of "ethics" is hand-wavy and ill-defined, because there's no professional organization.

Anyway, we can wish for good things and that'll party work and it's better than being cynical, but it'll only get you so far.

> I've never heard of this legal definition of "professional" that you put forward. Where is this in state or federal law?

Professional, as in professional engineer. Think civil engineers, for example, who need to be registered with the board of engineers in order to provide engineering services (eg. approving plans for construction, supervising construction).

Other countries also have specific legal statutes that regulate the title of engineer, such as Canada.

https://thelawdictionary.org/professional/

1. A person, who is a member of a professional body due to the education qualification and follows the prescribed moeal and prefessional code of conduct.

2. A person who has mastered a high level of expertise in a subject, notion on field.

Completely different definition. Dawn can refer to both a time of day and dish soap, but they are not interchangeable definitions.

> PROFESSIONAL MISCONDUCT 'Behavior different from what is prescribed in the moral, ethical and professional code of conduct'

For example only uses definition 1.

But when an individual says that they are a "professional", they could easily be using either definition, and still be using the word correctly. The OP said "Software engineers don't understand they're not professionals..." as if only the first definition was correct.

There's an incentive for lawyers, doctors, and accountants. You get a license to work from a board of peers, there's peer review, you can lose the confidence of that board and have the license to work revoked. I don't know the histories of those professions, and what's in common other than "obligations that go beyond self-interest". But I think lawyers are substantially different from other professions, even setting aside the state bar: they are officers of the court. They're an integral part of the legal system and they have a superseding duty to that system's function, not only to a client.

So how would software developers/programmers/etc create something equivalent to that? Seems to me some critical mass of that industry would need to lobby state and/or federal government for the creation of a board who then produces a standard and certifies members conform to the standard under penalty that if they do not police their own members that the state can revoke the franchise/monopoly on that certification.

“Attorneys, for example, can't engage in the kinds of advertising speech that would be entirely lawful if done by a car dealership or furniture store.”

Why does Saul Goodman come to mind.

> Attorneys, for example, can't engage in the kinds of advertising speech that would be entirely lawful if done by a car dealership or furniture store.

Are you sure about that? Because I've seen some really sleazy advertising by lawyers. Personal injury lawyers in general, but Jim Adler, the Texas Hammer especially comes to mind.

Just as a quick example, the Florida Bar had a regulation that an attorney’s website could feature a picture of the lawyer, but the background was required to be either a single or gradient color or a generic courthouse.

GP didn't say that law is not a profession, but rather that computer programming is not a profession. This is obviously the case, since there are neither educational requirements nor state-approved certifications for this work. There are ethical expectations, only because all humans have ethical expectations.

IANAL but one could certainly imagine 1A challenges to any attempt to enlist the state in the enforcement of a cartel like that of the lawyers or the physicians. In that sense it's no more likely that programming would ever become a profession than e.g. that writing would become a profession.

The word "professional" is overloaded. Perhaps a better term is "government certified professional". Medical doctors and (physical) engineers must meet a prescribed set of minimum standards and take an oath.

Attempts to form a set of software standards to be used in a similar way has proven elusive. Software is too malleable, and languages and "state of the art" architectures come and go like clothing styles (often without merit in my opinion). Physics and the human body don't change very often. Software approaches do.

I prefer accredited or registered.

Legally, being a professional only means you're exempt from the requirement for overtime pay.

Someone earlier in this thread is confusing "professional" and "engineer".

I think this is really critical part. In my country a lot of (private) banks promoted the title of clerks to 'officers'. Not the title employees asked, but being officers they are not entitled to overtime.

Agreed. There’s definitely a real technical meaning in the word professional. Although the idea of being a “profession” sounds good, it’s riddled with regulation and ways that make the job less likeable for the practitioners. I left one before becoming a software developer.

Nonetheless, being an ethical human being is just a good idea.

https://www.ieee.org/membership/qualifications.html

Search for, "Member grade". The only path for a large portion of all people being called "software engineers" would not even be considered for 6 years.

Those 6 years would probably be the most formative in educating them of their ethics and responsibilities. A service I am not sure IEEE really prioritizes anyways.

IEEE can do whatever it wants. They don't certify anyone as a professional engineer. That is NSPE's job in the US.

Yeah, I am apparently not qualified to receive anything based on experience. https://www.nspe.org/membership/types-membership

Or if I want to pay them to get a certification no one cares about, I can. https://www.nspe.org/resources/education/exam-review-and-pre...

The system in that regard is broken.

They're professional. They're not engineers, in the (legally) strict sense of the word. I think that's the distinction you meant to make.

"Engineer" as a title carries significant connotations of professionalism and integrity.

If computer programmers or other developers want to refer to themselves as engineers, then it behooves them to also step up their game in the ethical field.

Too often people want the "engineer" title because of its social cachet but don't strive to leave their mediocrity behind.

Are you implying that most software engineers aren’t ethical? That isn’t true in my experience.

I wholly object to this line of reasoning. I don't see how this wouldn't blow-back on software developers and make them an easier scapegoat.

Take these car manufacturers gaming the emissions test, for example. I find it very easy to imagine management making a series of requests for changes to the software that seem innocent but allow for this exact outcome: changing the way the system performs when being tested for emissions. They may have asked the development team to add some kind of a switch so that they could toggle the behavior on and off in order to assess how overall performance was impacted.

That sounds innocent enough to me that I wouldn't be eager to risk my job (and perhaps the ability to get work in the future) over suspicions. Certainly it's not enough that I'd try to get law enforcement involved.

This idea that software developers (or engineers, if you prefer that term) should be responsible for the entire system, even though they likely only work on one component, sounds to my ear like nothing more than setting those developers up to be easy scapegoats. If you wanted to get me on board in any way, I'd need to see some system where all code is signed and, perhaps, stored in some tamper-proof write-only storage system managed by a third party. I'd need to be able to verify beyond any doubt that code a particular developer wrote hasn't been tampered with before I'd be willing to send that developer to prison, or at the least, ruin their career.

Management is at fault for requesting the feature that enabled them to game the system. Management should pay the cost, I see no reason to cast any blame on developers. I have yet to see any compelling evidence that the developers knowingly added code specifically to game the system. If evidence of that is out there, I would be super interested in reading it and would appreciate any links anyone might have.

> I find it very easy to imagine management making a series of requests for changes to the software that seem innocent but allow for this exact outcome: changing the way the system performs when being tested for emissions.

Off the top of my head, I've written many pieces of code that 'game the system' but are used only for testing purposes. I know full well that if those pieces get into production they could be hazardous, but there is nothing ethically wrong with me creating them. If a superior takes that code and uses it, that's not my fault.

It would be supremely easy for management to put in a bunch of requirements for the VW emissions system that are blatantly dangerous, but 'for testing purposes'. The developers need to be able to trust the higher-ups and certainly shouldn't be held responsible for their decisions.

To clarify on my OP, I want to say that there is a distinction between a legal and ethical obligation.

In engineering only a PE really has the legal obligation. Their on the hook if something goes wrong. But at the same time everyone involved has an ethical obligation to ensure that best practices are being followed and nothing is being done that would result in harm. This is even if they wouldn't get in any legal trouble.

Or that a doctor has both legal and ethical obligations. Their ethical obligations can many times supersede their legal ones.

I am not arguing that we who write software need to be held legally accountable, but that we need to think about any potential ethical concerns with what we write. And if you are saying "well yeah" then we're in agreement. And I think this should be pervasive. I think we should frequently talk about it.

But we are in high demand right now. Many of us have the leverage to say "I don't agree with this" and go work somewhere else. If you personally have that ability I think it should be used as well. But I understand not everyone has that leverage.

I think I understand what you're saying and I agree that no software developer should willingly write code that will be used to break or circumvent the law. I believe that if I was put in such a position that I would walk rather than implement the feature. I don't think we are alone in that feeling, I believe many developers feel the same way.

I worry that a feature meant to circumvent or break the law will not be easy to spot. Many projects are broken into smaller pieces that interact, many times the laws or rules change from state to state. Even in regular day to day work I'll see a feature request that effectively undoes another request. Often developers are working in areas in which they are not subject experts.

I think it's easy to write code that could (or even will) be misused; it may not look troubling until it's interacting with the larger system.

But that's why I'm arguing that we have to think about it. As well as why I'm not arguing for legal accountability.

We have to be aware that the things we create aren't always going to be used for what we intend them to be used for. So you frequently have to ask yourself "how could this be abused?" That is something that engineers frequently ask themselves. I don't understand why developers can't be asked to do the same.

You are totally right, I honestly agree with you. It's an important question, it's worth asking and we should all be asking it. I think there's a lot more grey in software development, grey people may not have as much of in other fields. But that does not mean we can simply ignore the question and it certainly doesn't diminish it's importance. More than that, it doesn't diminish our responsibility.

My concern really revolves around the scapegoating and protecting developers from situations where there is more grey than black or white. But that is not a sheild from personal responsibility or moral obligation, the questions you raise are important in their own right.

I definitely think the ethical concerns are a little more abstracted with software than general engineering. I mean it is more obvious how a bridge affects people than a social network.

But because of that I'm not calling for legal obligation. But I'm calling for more people thinking about the abstraction and how what they write could be abused. But it is no easy matter. Ethics are hard.

You are right. From what I have seen that most of the time even quite senior engineers are not aware of larger implications beyond technical architecture of software they are developing.

Most engineering schools have classes about engineering ethics. When I was an undergrad in UC Berkeley, we not only had an ethics class (called "Social Implications of Computing") but also every single class had ethics sections, e.g. if you're working in a mission critical OS don't forget what happened to Therac-25, if you're writing ML Algorithms don't forget what happened to Uber etc... These were actual parts of classes, mentioned by professors. I think ethics is a very important part of both engineering and being a human being. You cannot make bridges and not be responsible if they collapse. You cannot make an OS and not be responsible if a bug kills bunch of people. Similarly, you cannot write fraud software and go by your day because that's what your boss ordered you to do. Engineers are highly skilled workers and we must have enough consciousness to deny doing a job if it's not ethical. Otherwise, it's just banality of evil.

>You cannot make bridges and not be responsible if they collapse. You cannot make an OS and not be responsible if a bug kills bunch of people.

This is a bogus comparison though. A bridge has one obvious use and can't be picked up and re-used in many other bridges. An OS (particularly an open source one) has nearly unlimited uses beyond the realm of the author's control.

Let's not put Linus in prison when a kernel panic results in a Tesla slamming into a wall.

That's not what it being argued here. Not by me and I do not believe by the parent to your comment.

One of my all-time favorites: https://en.wikipedia.org/wiki/Ariane_5#Notable_launches or how a bug can blow up a rocket ;-)

I like this. At University of Florida we had 1 class. It was...alright, better than none. Everyone graduating through the college of engineering had to take it. But to have it mentioned in every single class and the relevance would be better!

this is very encouraging to read, thanks for this comment

I had a similarly titled class at a no-name state school. I think the professor mentioned something about ACM recommending/requiring the course, so it should be pretty widespread.

On a more discouraging note, there was normally single digit attendance out of triple digit enrollment in the particular course I took. I'd say average engagement was ~6 hours total over the course of the semester... Including a movie. I don't know anyone that didn't get an A. There were a few from that class that went onto big names, and they weren't among the regular attendees.

I think some sort of ethical component to the degree is required for ABET accreditation. My CE degree was retroactively accredited so I'm not sure if current students have to take a separate class, but one of the changes that was made while I was a student and the ABET process was happening was that we spent some amount of time in our project courses' lecture time (each semester has a 5-credit project course) covering ethical issues. ACM/IEEE membership (which itself, or at least "membership/sponsorship of a professional society by the school", might have been part of the accreditation requirements too since we all joined IEEE) may impose additional requirements like discussion on ethics.

I think the general point stands though that for everyone calling for ethics classes -- a lot of CS and CS related degrees like CE already have them. Furthermore many companies already have various code of conduct types of annual training. The solution has to be something else than "education".

The other thing I'd add to this is if you look at the VW and other scandals, you'll see "rogue engineers" are the first ones executives will try to throw under the bus. Ethics aside, you're taking a huge risk in order to inflate someone else's bonus.

In the end if things go sideways the company won't have your back. That boss who told you to cheat (verbally of course) will suddenly have "no recollection" of the conversation.

Yes, I don't think this is the way it actually played out. As you mention, why would such a "rogue engineer" take such a risk for someone else? And why would that person risk sharing their culpability with a "rogue engineer" when I am sure they could phrase things in a way that sounds more innocent and reasonable.

> why would such a "rogue engineer" take such a risk for someone else?

It's well documented that people will often defer to authority, even if it's to do something they feel uneasy about [0]

Indeed, in my own experience as a younger engineer I had a manager sitting over my shoulder once dictating what I should but into an estimate, under my name. Indeed it was fairly innocuous stuff, and I'm sure he meant well, but still ...

In a business environment where you have all sorts of contractors, graduates, people floating through and a management culture of "getting things done" there are so many ways that this kind of thing can happen.

[0] https://www.simplypsychology.org/milgram.html

Sure, it won't be phrased that way and so obvious when presented. But I guess that's my point: Have a conscience, pay attention, and don't build anything illegal or unethical. Management may be sanctioning it now, but they won't have your back once the music stops.

As a rougue engineer I would expect at least an increase in my bonus which is proportional to the one of my chef. There is little reason to be a roughe engineer otherwise.

The usual reason is something like "you'll get fired if you don't".

Civil engineers also have professional associations and certification boards backing them up in most jurisdictions whereas we do not. We don’t have anything to appeal to when we go to our boss with ethical concerns besides, well, our sense of ethics. If the order has already made it down the chain, that sense is probably not shared.

FWIW, software engineers do have professional organizations like the IEEE and ACM, whether or not they back members in any significant way. There are also PE licenses although my understanding is that they are being discontinued for software engineering because basically no one was getting them in that field.

The thing that matters is not whether professional organizations exist, but whether their acceptance is mandatory.

The reason why this works in civil engineering is because the boss or the company cannot overrule the engineer. If the engineer declines to sign off some thingy as acceptable because X, Y and Z must be addressed, then that's it - the company can't proceed until the issues have been resolved; the management can't sign off by themselves, and they can't really even fire the engineer and get someone else to sign off for that because a complaint from the fired engineer about that would result in the process getting an independent review, (assuming the issues are real) the process getting stopped anyway, and the unethical engineer who agreed to sign off despite these flaws losing his/her licence - so that even if not signing gets you fired from one place, it's still the better choice than signing and being unable to ever work in that profession again. "Ethics standards in a profession" mean the standards that the profession imposes on its members even if they and their employer want to violate these standards.

Without effective enforcement, however, the whole process would be useless.

Of course, at least many of the mechanical engineers at auto companies most likely do have PEs. It's not just civil engineers. It's mostly any engineer/engineering manager who signs off on drawings, test, etc. for regulators. When I worked on oil rig designs, I started down the path to getting a PE and it would have been expected had I stayed in that business.

That said, I'm sure there would be lots of complaints if software had a similar system given that it would likely come with specific educational and other specific requirements.

Does all that happen often?

I don’t think it happens often.

Though Lawsuits are not uncommon in civil engineering. When I worked as an EIT (passed the state engineering exam, on way to becoming a Profesional engineer), our office did an analysis on someone else’s slope stability calculations, because the slope wasn’t stable and they wanted to figure out if it was done properly.

Civil engineering designs have a clear paper trail and the buck stops with the engineer who signed and stamped the drawings.

In many cases it is "you can over rule once", because that's all you're going to get. So no, it doesn't happen often, but the engineer legally has that power.

The implementation of this cheating is more insidious than you think. There could be a requirement to put in a "simulation" for testing. There's nothing unusual about implementing this feature. Then another request comes in at a later time to make this feature a build setting. Then at a later time someone who is not a software engineer toggles this build setting on production build during manufacturing to put it out in the field.

Hear hear. When building a bridge it's obvious when they're building a test bridge or a real life bridge for people to drive over.

"So you want to spend 100 million dollars on a bridge that you're only using for testing purposes. And you want to put this bridge right where it would be really useful to put a real bridge to handle rush hour traffic. And you have a deal with the city to collect toll payments for this 'test' bridge for the next 50 years. Yeah, I don't think we're going to be cutting any corners for the 'test' bridge."

"You want me to take 2 hours to add in a test feature that can be conditionally compiled in for testing purposes? Also I talked to John in testing and they actually want this feature for testing. Seems legit."

Two months later. "Hey, I found this test feature that would allow us to cheat on the regulations. All we have to do is set this compile flag to true. The programmers even made this neat UI that allows us to change the build configurations without having to talk to them."

In addition, I think it's much easier to go back into the field and examine a bridge to ensure that something important was constructed to the specifications that you required. This is not the case with most of the software people write, certainly in this case it's pretty difficult for a developer to pull a car of the production line and verify that the code they wrote made it into the car without any changes.

Counterpoint: Software engineers can just make it a configuration option and leave it to the user to decide how it is run. Unlike civil engineers who can't build a bridge with a configuration option for a 'light & unsafe' bridge, software engineers can make everything an option, and that's often the best practice.

Of course, assuming that they are unaware of how their user / client / employer will actually use the software, they should be fine.

"It should be noted that no ethically-trained software engineer would ever consent to write a "DestroyBaghdad" procedure. Basic professional ethics would instead require him to write a "DestroyCity" procedure, to which "Baghdad" could be given as a parameter."

--Nathaniel Borenstein

It's kind of hard to disagree...

True, no ethical engineer would write "DestroyBaghdad", but how about perfectly reasonable "set custom threshold for test" function that lets the client pick? And then the client chooses an unethical value.

I can't force my ethical standards onto an unethical employer without the help of a union, or at least a professional organization able to advocate for me.

Employer: Do this vile thing.

Me: No, that would be unethical.

Employer: Oh, you're right, of course. Train Steve instead.

Me: Okay.

[later] Me: Steve is trained.

Employer: Great. You're fired. Steve, do this vile thing.

There is another option.

Employer: Do this vile thing.

Me: No, that would be unethical.

Employer: Oh, you're right, of course. Train Steve instead.

Me: No. <quits>

So you always quit when asked to train someone? The reason Employer plays along in the beginning is so you don't realize Steve will be your replacement until it has already happened.

Besides that, some people will need to continue working as they search for another job, and training Steve could be parlayed into a rudimentary supervisory position on the resume. That helps if you ever intend to jump over to the developer-manager track.

Q: "Have you ever had anyone under you?"

A: "Yes. I was in charge of a trainee once."

Q: "How did that work out?"

A: "Great, until they gave him my old job, then fired me. Maybe we had a budget problem?"

Additionally, if you don't train Steve and instead just quit, that's not going to stop the Employer from asking Steve to do the unethical thing without training. Now the unethical thing is not only unethical, but it is also filled to the brim with bugs that cause unsafe situations for end users.

For what it's worth...

http://time.com/64707/mary-barra-gm-safety/

"The program is formatted to recognize employees who contribute ideas, or those who raise questions about safety issues before they become bigger problems. Call it an internal whistleblower program."

What does this mean in the VW case? I believe the unions are strong in Germany. So either the "rogue engineers" was a lie from the management or they were doing it in their own volition?

don't give up so easy. Think of Fair Trade coffee -- all it takes is a vocal minority to insist on a set of practices and that will come to be considered the norm. The social (and market) pressure of hiring only coders that have taken the "Oath of Shannon" or whatever will enforce the standards far better than any union.

Fair Trade coffee is not an inherently beautiful and good system. It has worthy goals and maybe does more good than bad, but it's hardly a great example.

It’s a metaphor. Nobody was forced to adopt it with any kind of governmental or economic coercion, just a vocal minority that thought it was a good idea to elevate the standards and do more than just buy the cheapest product in the market.

Wouldn’t fly in most European countries with reasonable labor laws, union or not. The company would be in deep shit.

I would say the U.S. has become blase about corporations not having a social benefit component. They can do really vile things, far worse than a human, and get off with a fine that may or may not be meaningful in inhibiting such behavior in the future. I can't think of a single historical case, which I think would be notable, of a state rescinding the articles of incorporation of a company thereby destroying its shareholders - the corporate equivalent of capital punishment.

Some states do have "social benefit" corporations as a thing, where their charter expressly states how they are to benefit society not merely make profit. But no state makes this kind of incorporation mandatory.

VW is a German company. Nissan is Japanese. VW engineers did their unethical thing in Germany. In the context of this story, your comment fails logically.

Well, AFAIK it weren't the VW people who build the cheating system and the ones who built it, pointed out that in some countries it might be illegal to use it. The VW people just decided (probably a manager) to activate the cheat system.

So I am wondering where an ethical standard could help in that chain. If you build international products its kinda hard to know every law on this planet and refusing to build something, just because someone could use it in a country where it is illegal to use it, could become very complicated very quickly.

I think the simplest solution would be to improve the test methodologies and maybe increase the penalties.

The cheat device that the VAG group added wasn't really in any sort of grey area.

This isn't like the wrong shape steering wheel, or accidentally too small wing mirrors.

This was a deliberately programmed system for cheating on emissions tests.

These things are pushed by management down, rarely from engineers upwards. The usual "solution" is to hire people from different cultures with lax laws, i.e. in WEU it's popular to hire EEU folks, in USA to hire Indians to do the dirty job, as they either don't understand the legal consequences or are from cultures used to "work around" laws.

> but active in fraud.

No we're not. We don't have societies that put any such responsibility and restrictions in place, along with benefits, for what we do. Whatever the company does with the code is the sole responsibility of it.

Like you said, other engineers have LEGAL obligations, what makes an obligation moral or not is extremely debatable and always subjective to the point of uselessness.

I'm not entirely unconvinced this isn't already illegal somehow. There's a lot of laws. I bet if you dig through the definition of "certification" for emissions standards you'd find some reason why everyone knowingly involved with a fraudulent submission is already subject to some legal penalties somehow already.

"It's hard having moral obligations and shit so why even bother with that"

Are you suggesting that ethics and morality are useless, and that software developers should not care about them?

No, he's saying that if he raises an objection, then his boss will fire him and find someone less competent to do the job. Then after he's fired for standing up for the good of society, society will tell him to starve to death because he doesn't have a job. Finally, his replacement will do a significantly worse job and the result will be greater damage to society.

This is why unions exist.

IANAL, but from what I can tell, unions tend to be organized primarily to protect workers, not the general public. Refusing to do unethical work is usually legally protected (e.g., by whistleblower laws), so your union representation can help you out (they have lawyers, and experience dealing with labor law) but it's not necessarily their main purpose.

When has a union in this context ever protected a member who refused to take action on behalf of the business for ethical reasons? Lots of unions protect inaction motivated by worker safety, but this is a different thing.

The -idea- of unions. They're not exactly in a strong place in the US.

I'm not. I'm saying that even if I were to be protected for standing up for immoral practices (which by and large I'm not) we're still putting the burden of morality in the wrong entity, the employee at the bottom of the decision tree. This is a burden that the state needs to impose on the top of the company directly, not have it be shifted to the bottom.

My comment, the OP, is not saying that the bottom of the tree should have the legal obligation. But I would argue that every branch should consider their own ethical responsibility. Don't confuse those two responsibilities.

Ethics isn't that simple. The tools need to exist and some people will use them for things we don't agree with.

The emissions software needs to be able to turn things on and off because the emissions requirements for certain jurisdictions and applications can be mutually exclusive. It just so happens that manufacturers flip switches they souldn't in order to fake compliance.

I've worked on software that can be used to make it much easier to drop a bomb on you. The main purpose of the contract was a software package for defensive applications (i.e. making it much harder for a car bomb to surprise a checkpoint). Should it have not been built because it has the potential to do things you or I might not agree with?

This is easy to pontificate about. How do you actually bring about culture change?

This is a great question. I would agree a lot with gnulinux here. I have a degree in physics and took some engineering class. In both the engineering classes and physics classes we talked about ethics, moreso in the engineering ones. We also had a required ethics course.

But I will add that another thing to help is to encourage it within your social sphere. This can easily be done by just asking your colleagues about their ethical concerns. You can also encourage this by talking about things that aren't your software. This helps everyone get in the frame of "I need to constantly think about ethical implications". Other people are talking about fear of being fired. While I think that has to come down to you and your ethical code/leverage, it is much harder to fire and replace a team. So I would actively encourage this kind of talk. Having worked as an engineer I can tell you that we constantly did talk about ethical implications of things that we worked on, things that our technology could be used for, and if other companies were doing ethical things. It's hard to draw the line sometimes, but the best thing I can think of is to constantly be talking about it and being aware that you have power.

Education. Engineering and science schools should have good and rigorous ethics courses. Taught by actual philosophers and engineers on the field dealing with ethically sensitive issues (like writing a real-time OS for an airplane, self-driving car etc). An engineer or a scientist shouldn't graduate without the ability to reason about the social and ethical implications of their jobs. Engineers, physicists, scientists, social workers change the world more than most individuals and thus they need the ability to reason ethically more than most individuals.

I think the real answer here is "legal obligation". You may sway some number of sw devs with classes on ethics, but in reality many will throw it all out the window to save their job unless there's a real and significant consequence in breaking said ethics.

You're mostly right. As I said to some other answer below, I think law enforcement is crucial to limit evil people doing evil. But well-intentioned people can do evil too, just because they didn't know any better. Engineers who made Therac-25 probably didn't wanna kill bunch of people, but they did. (Also read Hannah Arendt on how she talks about most Nazi officers thought 'they were just doing their jobs') Understanding that your job may have such implications is why ethics classes are very important. This shouldn't be optional, engineers should have the ability to be able to think about the implications of being an engineer.

Of course, I wonder if that would result in no-degree-me being out of a job :). I have worked in medical equipment and cancer diagnostics for 14 years now.

Well, I didn't mean to be an elitist. My point was that ethics is an important "skill" for engineers and you can train this skill by other means such as reading books. There are tons of books about theoretical ethics, scientific ethics, engineering ethics, bioethics etc. Even reading something as basic as Stallman can be very helpful for software engineers. I'm just pointing out that engineers should think more about ethics.

>Well, I didn't mean to be an elitist

No no no, I didn't take it that way. I was just remarking that I may find myself out of a job if regulation similar to civil engineering existed in our field due to the fact that I don't hold a degree. Nothing beyond that and it wasn't a response to anything you said.

I've argued to (and with) my coworkers on more than one occasion that we _should_ be personally liable in certain situations (we already have the FDA watching over us at a high level.) For example, if I were complicit in fudging the results of an inter-system repeatability test in my current position.

Do you think that teaching an unethical person about ethics makes them ethical? I never understood the point of ethics classes.

It’s hard to imaging that people working on emissions cheating would have been less likely to work on this, had they only taken the CS ethics class.

It's not about making an evil person not-evil. You don't have to be evil to make evil things. The point of ethics classes is to learn how to think ethically. How did old and modern philosophers think about ethics? If you're an evil person, then law enforcement should be there to discourage you not to be evil and hopefully penalties are high enough to stop you. But if you're not an evil person, and the law gives you certain freedom on ethically sensitive issues, then an ethics class will help you how to reason in such situations. When I don't debug my code properly sometimes I feel terrible because a bug can significantly change a person's day or maybe even injure them (I work in telematic industry). Being a competent engineer requires one to understand the implications of one's job and act accordingly which may be debugging properly etc...

Certainly not. But they can help sway people who are on the border to the ethical side of things. Ethics classes can also provide tools and mental models for dealing with gray areas which aren't as cut and dry as cheating on an emissions test.

Yes, and the the thing is that most people are "on the border" - that is to say, little more than a product of their environment (speaking of their actions/attitudes, not their worth as people). If you teach an entire population ethics (assuming it's done well and everything that comes with that), you will still have your 5-10% of rotten apples, just as you'd have 5-10% who would seek out ethics on their own. The big difference will be in the 80-90%.

Like most classes, the point is to educate people about something they don't know about, broaden their horizons and perhaps cast new light on apparently familiar topics.

I've never understood the point of computer science, I can produce a perfectly functional BASIC program.

Ethics has its own vocabulary, history, and arguments in ethics are common and often don't have a resolution other than compromise. It's predicated on making logical arguments and has ties to epistemology: how do we know what we know?

So saying you don't understand the point of ethics classes, as if people are ethical or are unethical, is like saying you don't understand the point of teaching logic or debate as if people either pop out of childhood with an innate conformance or resistance to one kind of logic or debate, no other kinds exist and there are no arguments.

If you took an ethics class and don't understand the point of it, then I'd argue it wasn't a good ethics class.

> We are writing things that affect peoples lives. That could kill people. That affect the environment. We need to hold ourselves to the same moral obligations as other engineers. We have a duty to prevent harming people when we write our software.

High and mighty ideals, but come on.

If you're working on software for a medical device, then yes.

But on another post today it was pointed out that crypto currencies are burning more electricity than Iceland.

That's straight out environmental vandalism. Is everyone involved in crypto expected to down tools, cos ethics?

Nobody told me about the code of ethics I have. Can you send me a link? Is it somewhere in the windows 7 EULA I can never get myself to read those darn things :)

Well in this case it was factory testing and had nothing to do with software.

In the case of VW, I highly doubt those software engineers were told plainly what they were doing. While I agree they have an obligation also keep in mind speaking up would probably be career suicide.

The liability is on the executives that make take these illegal actions.

I understand your sentiment, but I think this only applies if you want to be an especially good person. The rest of the professional world does not have to match such a high standard of ethics, and as much as I would love for everyone to have to behave nobly, it's pretty unreasonable to ask software engineers to report software that exploits loopholes in regulations while our counterparts in banking, private equity, law and medicine basically live exclusively off of doing that.

edit for the downvoters: in this exact case, the executives of the companies hacked the system, their lawyers probably approved of it, and the strategists who conceived of it likely used to work as investment bankers. And yet, the software engineers are the ones expected to take the high road?

> while our counterparts in banking, private equity, law and medicine basically live exclusively off of doing that.

Literally every one of those professions are governed by legal ethical requirements. Software engineers/devs are not, which is kind of the whole point.

PE doesn't any more than programming. Banking's laws are just moats for the established players masquerading as regulations.

Law has them, but in a strange twist, their ability as lawyers make it so that they don't matter. And medicine is too complex for me to really comment, but from my vantage point, it looks like they're free to be scumbags so long as they claim they won't be when they get their degree.

But every one of these are largely just regulated by the idea that if you're too big of a scumbag, the feds will nail you for wire fraud. That seems to be the only real rule of law or code of ethics we have in the professional world. And software engineers have to obey it as much as everyone else.

>>PE doesn't any more than programming. Banking's laws are just moats for the established players masquerading as regulations.

PE sure, I probably should have noted I know little about that, but when you talk about banking you're now going from "have no ethics laws" to "well, they exist, but they're not effective in my opinion."

>Law has them, but in a strange twist, their ability as lawyers make it so that they don't matter

Lawyers get busted on ethics violations all of the time. My mother is a lawyer and for a short time I worked as a pseudo legal secretary in a mid sized city. I recall numerous lawyers being disbarred for ethics issues. I have no idea what the real numbers are, but I don't think you do either.

>And medicine is too complex for me to really comment, but from my vantage point, it looks like they're free to be scumbags so long as they claim they won't be when they get their degree.

So you don't really understand the issue, but you've heard some stuff on the news and maybe from friends, so you write the entire things off.

How about civil engineers who are personally liable for their work?

>But every one of these are largely just regulated by the idea that if you're too big of a scumbag, the feds will nail you for wire fraud. That seems to be the only real rule of law or code of ethics we have in the professional world. And software engineers have to obey it as much as everyone else.

Again, moving goalposts. If you don't feel the current regulations wrt doctors, lawyers, and bankers are enough, fine, many would agree, but that's not the issue here.

>PE sure, I probably should have noted I know little about that, but when you talk about banking you're now going from "have no ethics laws" to "well, they exist, but they're not effective in my opinion."

Oh, if we're going to be pedantic about what exists, then software engineers have them. There are computer hacking laws! And software IP and copyright rules. See, we're regulated, too. I thought we were operating on the premise that the existing structure was insufficient or irrelevant to any meaningful end, but since it technically exists, I'm not sure why you're arguing that we need to create them.

Honestly, your entire reply seems like you're going out of your way to miss the point. When you made a mistake about PE, I didn't concoct some strawman to try to embarrass and highlight your ignorance, so I don't know why you're trying to do that to me. A software engineer's counterpart isn't some ambulance chaser getting disbarred. Biglaw associates are the professional counterparts to software engineers. And in medicine, I've seen tens of thousands of extra opioid deaths and hundreds of billions in insurance/medicare scams attributed to the medical industry to have a bit of an opinion at least.

If you think the argument that industry-specific regulations don't matter, and that the only regulations that matter are federal fraud laws is a goalpost change than I don't know what to tell you. My argument is that the industry codes don't matter, and that any semblance of actual regulation takes place at the federal fraud level, so putting an unnecessary step of internal regulation is pointless.

Reply on news.ycombinator.com