Why do we never hear about programmers ethically challenging the auto makers when they clearly know about these flaws? It's akin to working on medical devices. Where is the incentive to not make lethal mistakes?
Many programmers, though, will do whatever they're told, even when they know it's pointless, wasteful, or dangerous. I've read literally hundreds of stories about programmers working on something that was egregiously fucked up. But I've heard damned few stories where programmers refused to do something they thought wrong and either a) got management to back down, or b) got fired for their refusal to do something unethical.
I don't blame individuals for that. We're not trained as professionals, we don't have a professional organization that most of us join, and we therefore don't have a serious professional code of ethics. It's unreasonable to expect most people to make ethical stands alone while creating a code on the fly. But I'd love to see the situation fixed.
I'd more thinking about a professional organization like the AMA, the assorted bar organizations, the society of professional journalists, or the various engineering societies. Basically I want an organization that can set forth reasonable professional ethics, do advocacy and education, and act as our collective voice to back up individual engineers who refuse to do something unethical.
Companies like Toyota generally outsource software development to other companies like Bosch. Bosch employs a bunch of programmers in Bangalore who are just out of college, and pay them peanuts to develop a safety critical system.
* they are part of a big enough team that individual responsibility is removed
* they're not given enough time to produce good quality code
* the design came from someone without sufficient expertise, and the programmers "just built it"
* they did the best given the time they had
* it passed testing
* programmers are generally not empowered to make go/no-go decisions
* even if they did express concern they may not have been listened to
* any internal challenges on ethical grounds were probably kept quiet and worked around
Because we know how that always ends, with the person driven out of the company or set up to be fired.
The only way to get big corporations to behave is prison time for executives so they become paranoid about doing things right. Unfortunately in the USA we never ever seem to do that, we just give them a few million dollars in fines instead which they pay with the next month's profits (ala BP).