Software Development Ethics
theprogrammersparadox.blogspot.com
theprogrammersparadox.blogspot.com
At his Khoury College graduation celebration a week ago, this is the first time that the following oath was recited by the graduating class:
KHOURY COLLEGE OATH FOR COMPUTING
Today, I join the ranks of computer scientists worldwide.
I will remember that I remain a member of society, with special obligations to all my fellow human beings.
I will design and build computing systems that enhance the quality of daily life for individuals and for society.
I will protect the dignity of users and others affected by computing systems, respecting the diversity of all cultures, and safeguarding against threats to health and safety.
I will respect the privacy and rights of all people and recognize the special role I have in judiciously collecting, storing and using their information, and creating systems that aim to shape their behavior.
I will work for fair wages; honorably guarding my reputation and my colleagues in our work practices, while respecting the intellectual contributions of others.
I will improve the public understanding of computing and its consequences.
May I always act to preserve the finest traditions of my field, and may I long experience the joy of inventing the future through my endeavors.
What does this mean? What's an example of an unfair wage?
But if you see your colleague doing as much as you do and more for $600 then neither of you is working for a fair wage.
What about open source contributions. You are giving away code that you worked on for free.
What about building evil things for good people? Building good things for evil people? Who gets to write the source code of the `isEvil(foo)` function, and can I fork it if I disagree?
The section on automation seems deeply confused; first it says that automation must always be manually triggered by the people involved, then in the next sentence it provides guidance on scheduling cron jobs, which would seem to be ruled out by the manual triggering rule. Signing up users to an email list automatically and without a way for them to remove themselves is presented as an example of the kind of evil automation this would prevent, and I agree it should be prevented as I don’t like that either, but it goes on to demonstrate that the proposed rule also inhibits things I think are okay, like if I authenticate with TouchID I want my Mac to sign in with whatever credentials it has at its disposal.
The second section on never lying to or tricking users is similarly deeply flawed; it is motivated again by a good example of disallowing unreadable EULAs that sign away personal data, but again goes on to demonstrate its own overbroadness - which reaches so far that it absurdly claims that not solving the cache invalidation problem is a moral violation.
The third section is somehow worse still; in trying to make a developer ethically liable for anything anyone does with their code, it positions itself at odds with several of the most common and venerable open sources licenses that vast chunks of software have been built on and released under. Aside from the fact that this is simply unworkable on a level even Stallman would balk at, “opposing some of the most common licenses” should have clued in the author that their document does not align with most software developer moral codes.
Ethics is not about right versus wrong. That is morality. So for this conversation it does not matter if a given software is evil, a given client is evil, or if anybody cares that any of these are evil. Evil is irrelevant and highly subjective.
Ethics is about conformance to rules that govern practice and behavior. In industries that observe ethics the violations of such rules don't just mean you get fired. It means you can never do that job for anybody ever again. The goal is to ensure that people practicing in a given field of work prioritize the quality of work above that of profit or employer demands.
Even ignoring the fact that the article itself defines its rules in a moral context, I feel like GP's concerns over the content are equally valid even just as professional rules. I thought the part about cache validation was hyperbole, but no, it's literally right there in the article: "Stale stuff in a cache would be an ethics violation". I don't really see how this would be a remotely reasonable rule for an industrial ethical code that you describe. If we're putting "stale cache results" in the list of rules where breaking them doesn't "just mean you get fired", but that "you can never do that job for anybody ever again", we might as well just all just retire as software developers and take up a new trade; even the best programmers in the world accidentally write buggy code sometimes, so we'd be better off finding some other job on our own terms rather than waiting for inevitably getting "disbarred" (or whatever the equivalent term here would be).
First result claims morals are your sense of right and wrong, ethics are the good and evil agreed upon by your community. [1]
Second and third results claim “morals” and “ethics” are used interchangeably. The second says even philosophers of ethicists consider them interchangeable. Both of them then add that if a distinction is to be drawn, they agree with the first. [2][3]
The fourth result says the opposite, that ethics is an “individual assessment” while morality is an “intersubjective community” evaluation.
I don’t think the terms are as cleanly separated as your comment suggests. I also firmly disagree with this other claim that the “mere concept of ethics is utterly foreign and unknown” to software as an industry.
1: https://www.verywellmind.com/morality-vs-ethics-what-s-the-d...
2: https://www.britannica.com/story/whats-the-difference-betwee...
3: https://www.diffen.com/difference/Ethics_vs_Morals
4: https://theconversation.com/amp/you-say-morals-i-say-ethics-...
Historically, "morals" was more or less Cicero's translation for "ethics". That's probably the clearest distinction between the terms you can find.
It's whatever is trendy to say or think at the moment. A set of arbitrary unenforceable rules to shame practices that is decided by whoever convinced other ethicist to grant him a PhD.
Of course, all that is done without going through the rigor of making these practices illegal. That would have a real impact. And require real work.
> The section on automation seems deeply confused
It makes no sense at all. The author claims to be technical but honestly it reads as something someone who never programmed anything would write. Just knowing the "jargon" and trying to sound like they are more knowledgeable than they are. I noticed "Ethics of X" is generally a great field for those who can't cut it in field X. See the "Ethics of AI crowd as an example", that seems to be mostly made of "tech adjacent" folks. Probably the best example here [0].
> make a developer ethically liable for anything anyone does with their code
That's plain nonsense.
[0] https://syncedreview.com/2020/06/30/yann-lecun-quits-twitter...
Say you have a decompiler like ghidra or a dissasembler like x64dbg even. These tools are heavily used by malicious actors AND the people working actively against them.
Would creation and updating of such tools be "ethically neutral" or would we ask the creators of such tools to be held "ethically accountable" for what bad actors do with them ?
That would lead to the age old bad situtation, that you have the bad guys with the tools and the good guys cant do anyting about this as this type of research would be considered ehtically questionable at best.
- Someone, somewhere
One person's 'freedom fighter' is another person's terrorist.
These ethics guidelines strike me as well-intentioned, but naïve.
"If you write something innocent and later find out that it is being used for evil, you are now obligated to do something about that. You can’t ignore it or say you didn't know. You are now stuck and have to do your best to make that situation better."
If you create a library and a state-sponsored hacking group uses it, what are you expected to do? Call them out? It is not like other users of your library would associate you with the hacking group. Anytime a general purpose tool is released a devious creative mind might find a way to use it that the original creator did not think of.
True, but tools are becoming more sophisticated and different people have different definitions.
Sophisticated tools like large language models have many capabilities that their creators did not envision.
Different definitions mean that one person's 'doxxing dissidents' is another person's 'fighting domestic terrorism'.
Good tools are easy to repurpose for many uses and causes, both good and bad.
A sword can be wielded by a knight to defend the weak and innocent and serve king and country. The same sword can brandished by a pirate to kill, plunder, and destroy. Should we ascribe the good or bad done with the sword to the swordsmith who forged it? I say 'No.' The bearer of the sword is responsible for how the sword is used.
Only if there are ethical uses for such tools.
One could make a hypothetical argument that if a known terrorist planted a bomb that would kill thousands of innocent people and the terrorist was caught before the bomb exploded it might be considered "for the greater good" to use extreme measures to find out where the bomb is and how to disarm it.
Yes, it is a contrived example, but some nations have had to make similar decisions. I am not saying that I agree with such choices, but I have never faced such a decision. It's hard to know what the best answer is in such difficult circumstances.
Is it right to torture people? No.
Is it right to let innocent people die? No.
So what is the ethically correct choice?
Which is a fun scenario to contemplate for entertainment, but decades of research and a fair amount of rather upsetting empirical evidence shows that torture does not actually work.
All those stories where torture is used to get information? Just stories.
Jack Bauer? A fictional character.
If you want to get information out of otherwise-hostile people, you need to make a connection with them. You need them to trust you.
Torture is only good at establishing dominance and breaking people's spirit. People will tell you anything they think you want to hear to stop torture, regardless of whether it's true.
> So what is the ethically correct choice?
Easy. Don't torture people. It has no positive uses.
All of these have very legitimate uses, and all of them have been used for torture.
If you read about medieval warfare you'll see knights often used their swords on the weak and innocent at the behest of their lord. If you were a medieval blacksmith making swords during wartime, a good percent of them would end up being used for what we today would consider atrocities.
Not that there weren't ethical uses for swords too.
Yes, but there's a "know it when you see it difference" between most real life activity and straw men, too.
The less like straw men the arguments are, the harder they are to discern from reality.
https://ethics.acm.org/code-of-ethics/software-engineering-c...
To wit:
> First, no matter how many instructions are executed for any type of automation, the code must always be triggered by humans. Not someone, or anyone, but the actual people involved.
The author is against traffic lights?
> A big part of this is that the data presented to the users must be correct, at all times. So, it must be named correctly, modeled correctly, stored correctly, cleaned correctly, and presented correctly. Stale stuff in a cache would be an ethics violation. Badly modeled data that is broken is one as well.
The author would in all likelihood have to turn themselves in - and besides, who gets to decide the meaning of words like "correct" or "badly modeled"? Also, how does this work with things like eventual consistency? (Remember, reality itself is eventually consistent!)
> The third part is to not enable chaos, strife, and discord. This is the hardest of the three.
This is indeed hard. However, consider how far-reaching this statement is. Chaos, strife, and discord are caused by human actors with intent -- we should now include telegram (the old kind), telephone communication, physical letters, etc.
I would be curious to hear the author's opinion on mass surveillance to enforce all these rules, as that would seem to be necessary if you were really going to to do this.
That said, these concerns are not new. When someone creates technology, it is almost guaranteed to be abused, even if that was not the creators intent.
Lawyers have an actual defined code of ethics that they have to follow or they loser their license, defined by people whose job it is to make rules. Even theirs isn't this draconian. I'm pretty sure I don't want to be bound by somebody else's code of ethics and I know I don't want to be bound by this guy's.
> So, for in-house batch jobs, they are started and stopped by operations personnel. Someone also has to explicitly schedule them to run at a frequency. Those people are the ones that ‘operate’ the software and provide it as a service for the users. They are on the hook for it, so they need to be able to control it.
I used to be one of these folks. I now write operational software that operates other software, but I was the guy pushing the button. For one, if you're the button pusher you probably know very little about what the code actually does. You're often more familiar with how it behaves on a system, some high level KTLO metrics, and some basic runbooks. To shove all of the responsibility for software outcomes on them is problematic to say the least. Historically, Ops folks are also some of the least paid people in tech, aside from analysts and the position is largely outsourced these days.
People like this should not write or participate in ethical codes. Ethical codes are high level frameworks that involve processes for reporting violations, not some rigid structure on how "everything needs to be done this way". They'll provide criteria to abide by that helps readers ensure they are following the system.
I work in a lot of highly regulated industries that involve governments; the code for how to treat government workers and adjacency is just that and necessarily that. The reason is that corruption is not black and white, you can't just name off familial relationships and hope you've covered everything. Instead you have to refer to things like undue influence. You also can't just name off "gifts"; for instance, if the company comps something due to a bad experience is that a "gift"? Of course not, but a sufficiently rigorous ethical code would make a world where refunds were controversial. All of these things also come with a responsibility to document your tracks and operate in the clear. The minute you stop operating in the clear you might not be charged with a crime, depending on what a reviewing committee can uncover, but you'll certainly be out of a job.
For this, I think something like the IEEE should be building a code of ethics for software. They have the acumen and are industry-aware enough to sort through various contexts and make sure the code is verbose yet light enough to stand the test of time without incurring prohibition style statements like, "let's bring back operators".
If something is actually a problem for citizens, ask your legislators to make a law encompassing fair use and other compliance standards. Don't just make up new ethical rules and pretend barriers because you're jealous.
If you thought of and executed ChatGPT, or Tindr, or Facebook, before their creators, would you still have the same "ethical" views that you claim should be followed?
Also reading through the article, some of the author's propositions are insane. For example:
> So, for in-house batch jobs, they are started and stopped by operations personnel
No, the whole point of some of our work is to increase automation and reduce opex costs, so that companies don't need to hire people because people are expensive and risky.
> Second, a computer should never lie or trick the users
This requires users to be informed on what their device is doing. How many people actually read the terms and conditions of the services they use? If they can't even be bothered to read that, what's to say they're going to read another long, boring document that explains what every feature of your app does so they cannot claim they were "tricked"?
Sure dark patterns are not very fantastic - we can absolutely address those. But creating an informed user-base is very challenging.
Publicly, the purveyors of all of the above purport to have ethics.
OpenAI's models have safeguards in them against discussing some topics and providing certain information.
I suspect it's just a social survival strategy for these companies, nothing more; not an actual commitment to ethics.
Typically it is not someone just making them up out of jealousy, it is an organization that comes to consensus about how they want to be viewed as a group. This is a good thing.
It is important to have both.
A big part of the reason no one reads them is because they're a giant legal document. If you want to inform your users you could say something like "Hey this is what kind of data we collect and what kinds of people we share it with" it doesn't cover everything but it gives people a much better idea of what they are actually agreeing to
Therein lies the rub then. People can still claim they're "lied to" because there was no written document explaining something a particular service did. Which brings me back to my original argument, some of OP's claims are extremely difficult to actually implement.
Do you mean in terms of developing the services, or in terms of scale? If you're talking about scale, I can assure you there are many people would never be confronted with that scenario since their ethics would have held them back. There is a world of difference between being successful and being a titan. Even though anyone can have questionable ethics, amassing power over others virtually ensures that one has questionable ethics.
We do? Says who?
Why don't people who make hunting rifles or kitchen knives need this ethical code?
Cars can run over people on purpose; why do auto makers not need an ethical code about that, only about safety?
OK, let's step away from the strawman and discuss only software whose only conceivable use is killing people. Is that automatically evil, though? Say that your country is at war and your government orders you to write such a thing. If you don't you could be tried as a traitor.
An automotive engineer doesn't lose sleep at night over the idea that someone might sharpen a piece of a chrome fender using a grinding wheel, and then use it to cut someone's throat. Or strangle someone with a fan belt.
That's exactly what this blog is asserting though.
I would say that not only doesn't reflect sound ethics, it's not an example of good mental health.
But couldn’t “doing something” be a license change going forward. Seems like a reasonable application of ethics.
I don’t see a huge issue here.
I mean that is the issue. There's nothing novel or insightful about this piece on ethics in the least. "Hand wave, hand wave, software engineers should be obliged to know about and actively mitigate all the downstream effects of their software, but only if it's evil". Yeah, okay, move along.
Words in a license will not stop an evil-doer.
Ok. What is this evil you speak of, person who wants me to think I'm like you? I don't get it. Define it clearly please.
(No definition follows.)
> So we, as software developers, need an ethical code
Oh, so because of <undefined thing> we need <ethical code>? Why is that, then? Please explain all the whys.
(None of the whys follow. I guess we're not so alike after all. Drat!)
I believe we as a cohort, a group of similarly-titled people, a group seeking possibly similar things should have a code of ethics. I really do. But not because of no reason and "so there!"
What separates most of us from actual engineers is not just an iron ring, but a code of ethics (well, nowadays, also common scholastic training - can I get a "hell yeah" from the code bootcamp folks?). Most guilds used to have those codes, in fact. And union membership is predicated on agreeing with the union's goals. And I could never quite agree with the Cub Scout oath or the twelve steps' higher power, but those serve the same purpose, too, and I admire them even when I don't agree.
I think I disagree with the post's first reply, from "anonymous:" "really well written." There's not enough thought before the writing started. I think it deserves a little more of that and then probably a bunch of editing before it's worthy of a real takedown. And being worthy of a takedown is a badge of honor. (Honor being related to ethics but given ethics isn't defined, I won't bother either.)
The better question: Is it time for us to adopt some code of ethics?
Many other professions have important ethical standards. Medicine, law, civil engineering, and many other fields. These standards are broadly respected by the public.
Surely by this point we can all see that software is often just as critical as bridges, medicine, justice, and other public goods. Should we not hold ourselves to some standard?
What is our profession? You've hit on an important point there. What?
We, most of us, write code, but what industry are we part of? And there it all gets hazy.
It's like defining ethics for people involved in moving vehicles or ethics for people who write text in a given language.
[1] http://www.wfeo.org/wp-content/uploads/code_of_ethics/WFEO_M...
------
In the course of engineering practise, I will endeavour to:
#Demonstrate Integrity
- Refrain from fraudulent, corrupt or criminal practices Be objective and truthful
- Practise fairly and with good faith towards clients, colleagues and others
#Practise Competently
- Practise in a careful and diligent manner in accordance with my areas of competence
-Practise in accordance with accepted engineering practices, standards and codes
- Maintain and strive to enhance the body of knowledge in which I practise
#Exercise Leadership
- Practise so as to enhance the quality of life in society
- Strive to contribute to the advancement of the body of knowledge within which I practise, and to the profession in general
- Foster the public’s understanding of technical issues and the role of engineering
#Protect the Natural and Built Environment
- Create and implement engineering solutions for a sustainable future
- Be mindful of the economic, societal and environmental consequences of actions or projects
-Promote and protect the health, safety and well being of the community and the environment
AI will address that. Smaller talent can write more via prompt-drive code generation and refactoring; it just has to check that it's not garbage.
> assholes who will happily build slaughter bots
Chances are you live in a country that owns fleets of machines which fit that description, out of the taxes that you pay, under the direction of the government you elected.
Basically every job that is called a "profession" is explicitly expected to refuse their paymaster's demands when they are unethical.
There can exist code whose only purpose, only effect is to perpetrate some harm or break the law.
This article is saying that even if you hear that some innocent code of yours is being used in an evil program, you are obligated to do something.
I mean, that even goes against free software. If I put out some innocent GPL or BSD licensed thing, and someone uses it for evil, and then I try to use my copyright holder position to wrangle the use out of their hands, it is no longer free software.
Oh, and don't forget that non-free, closed-source software is what is unethical in some circles.
This person doesn’t really care about freedom of speech. It is easy to define speech you disagree with as either lies or hate speech.
In fact, as others have pointed out in the comments, there are a decent number of logical inconsistencies in this article and so this article has untruths and therefore lies and should be suppressed by ethical software. It also characterizes some people as“evil” which in my mind is also hate speech and should therefore be suppressed.