Comment by schaunwheeler | Hacker News Reader

Comment by schaunwheeler | Hacker News Reader

I find that people in tech, especially in my own field of data science, treat ethics as a relatively simple problem. If they were building a technical product, they would at least make an appearance of checking assumptions, and worrying about breadth and depth of stakeholder buy-in, and building tests for unintended consequences. But when it comes to building an ethical "product" to guide their work, it's like all of the best-practices go by the wayside. For example:

https://www.wired.com/story/should-data-scientists-adhere-to...

Data for Democracy came up with an ethical code for data scientists and is now talking it up asking people to sign on to it. The thing is basically the product a few months of working groups plus a day-long hackathon, and it's already been put out on the market, so to speak. So it's not surprising that they produced something that could, in many ways, actually run counter to their goals. I firmly believe the ethical code as written is itself unethical:

https://towardsdatascience.com/an-ethical-code-cant-be-about...

What I've found amazing is that the community that built the D4D ethical code has been entirely unwilling to invite criticism of the code, or even engage with those who question it. The fact that willingness to entertain criticism and consider unintended consequences is actually one of the pillars of their ethical code makes it doubly troubling.

I think the “problem” is that ethical questions and nuanced thinking generally get in the way of “getting things done”. It seems that the software/startup/disruption community glorifies moving fast, shipping an MVP and then iterating on improving measurable metrics. None of these are conducive to dialectic and thinking carefully about consequences. Heck, the popularity of weak/dynamic typing and REPL-it-till-it-works shows that many programmers today aren’t willing to reason about effects even within their own code... leave alone societal consequences! Nothing wrong with that style of programming as such, but I think it’s a symptom of the ratio of thinking-to-doing that many find comfortable.

Software is unique in that it gives us the flexibility to fix bugs after shipping, unlike any other engineering field. Over the last few decades, we seem to have gotten increasingly carried away with emphasis on that feature. Also has to do with the incentive structure encouraged in SV.

I agree with this. There comes a point (much sooner than most people like to recognize) when quality and safety get sacrificed to just getting things done. However, I think that's a manageable risk. Most focus - and tooling - centered on analytic/technical design rather than implementation has the potential to catch problematic implementations before they're shipped.

https://towardsdatascience.com/data-is-a-stakeholder-31bfdb6...

It's not fool-proof, of course, and I think a change in perspective away from just moving fast and breaking things would be quite healthy for the industry as a whole.

I think the problem is, you can fix technical bugs after shipping. It's much, much harder to fix ethical bugs.

Exactly. The cost of fixing a bug after the product is out the door is much higher when the bug is ethical rather than technical. Therefore, don't move the product out the door so quickly. Ethical QA takes time.

I’ve been saying this a lot lately, but software needs a professional associstion (like the AMA) with teeth. Ethics are not easy, and without collective governing power, they simply won’t exist.

https://news.ycombinator.com/item?id=16423328

https://news.ycombinator.com/item?id=16420659

I disagree. Not in principle, but because I think it won't work. If an organization has teeth in this market it will be turned into a weapon to gate out or remove competition.

What makes software so different from every other endeavor in that regard?

Scope

Medicine encompasses everything from practice to research, humans to non-human animals, procedures and devices to drugs and therapies, and touches nearly every human alive.

Engineering is even more diverse and penetrating, and software is merely a subset.

It ain’t scope.

If I understand your question: nothing, professional associations become anti-competitive gatekeepers in every endeavor.

Roughly 30% of US workers have mandatory occupational licensing before they can do their jobs. Licensed services have roughly 15% higher prices than nonlicensed ones. Most of those workers are not in safety critical jobs - on average an interior designer faces more stringent licensing requirements than an EMT.

Unions and professional associations have consistently been the main lobbyists advocating professional licensing as a way to keep wages high (by keeping supply low) and guarantee a role for the organization. Even in medicine, the number of residency slots is controlled in a way that creates a stable, minor shortage of doctors.

I'm not against all licensing and oversight, obviously. I'd rather have a cap on residencies than untrained doctors, if those are the only two choices. But the history of 'every other endeavor' looks to me like a lesson in how oversight organizations will cause exactly this problem.

There is always a trade off, but each occupation learns the hard way thst it’s worth it. Hopefully software will figure that out in time to regulate itself before the government steps in with agendas and a heavy hand.

So... competition > ethics?

Ethical software with less competition seems better than unethical software with more competition, eh?

I worry that Ethical software will lose out to unethical software. There is too much pressure and low-hanging fruit to do any kind of gate-keeping that won't be circumvented.

If you want to take the required measures to not circumvent the gate-keeper, that is strangling the entire sector. Since most developed countries see this sector as the basis for future growth, so doing that seems unlikely. Beyond that, the growth would probably move to less restrictive places.

I suppose the question is how much faith you have in oversight boards to actually produce ethical outcomes.

My worry wouldn't be "they'll cut down on competition to ensure ethical behavior", it would be "they'll block competition and protect their member instead of ensuring ethics."

I think the top-level essay here is extremely cogent: oversight is almost always limited to ensuring competence. In medicine, that gets us 'ethics' when ethics means "providing good care and avoiding bad care". (It hasn't gotten us ethics when the question is anything complicated, like defining mental illness.) Competence in data science and software is largely orthogonal to ethics, so I expect an oversight board would achieve very little in that regard.

A think a governing body solves the wrong problem. Most of the ethical issues that have arisen around data science in particular are actually competency issues. It's not that someone deliberately set out to target already-vulnerable populations. It's that they designed systems without building in checks for unintended consequences. A governing body accomplishes what any legal framework accomplishes: it provides instruction and incentives to limit liability. A legal framework is different than an ethical framework. What we need is an ethical framework.

The original Hippocratic Oath did a fairly good job of this by stipulating ways that a doctor could prove his competence. Doctors who adhered to the oath weren't better doctors because they had some kind of internal moral compass or external adjudicating body. They were better doctors because only the doctors who had competency to spare were willing to make the sacrifices that adherence to the oath required.

https://hackernoon.com/on-the-difficulty-of-creating-a-data-...

Ethical problems get solved (as much as that type of problem can ever really be "solved") by individual practitioners refusing to work with other individual practitioners who refuse to adhere to some basic best practices. That creates a network of competent and trustworthy individuals, who are still totally fallible, but who put their own practice up toe constant public scrutiny.

Ethical problems are problems of systemic risk. Systemic risk doesn't get solved through organizations.

The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge.

i want to stress that i'm not challenging the person, just the text.

> The governing body would not just be there to arbitrate, advocate, or negotiate. They would also be there for promoting training and sharing of knowledge.

this reads like duct-tape solutions. it's always the case that a governing body grows and grows and gets in the way. "it's not just there to XYZ, it also BLAHBLAHBLAH".

a governing body is one possible solution to the problem of people not understanding ethics (or how to figure out what ethics are), that I don't think will solve the larger problem.

a governing body will need to be run by practitioners of governance, and enforced by practitioners of enforcement.

ethics needs to be practiced everywhere, all the time. and a governing body won't make that happen.

people need to be learning about why their activities are "good" or "bad" or "questionable".

in my (simplified version) view: we're not lacking incentive to work (money!), we're lacking incentive to not work (people shouting at us for being assholes for doing asshole work [e.g. bombs, baby killing, evil algos]).

we need to cultivate behaviors of ethical culture... instead we're cultivating behaviors that promote short-term-ism (buy your way to happiness, bomb your way to safety, build your dream home on loan!, opportunity cost services saving us from having to cleaning up after ourselves, trash removal services that vanish things away to faraway dumps we never think of)

I honestly don't see what's "duct-tape" about suggesting that, being our organization, we'd get to decide how it's shaped.

I don't think training and sharing of knowledge is the problem. The problem is creating an appropriate incentive structure for maintaining trust in individual practitioners' competency. That doesn't happen in a top-down way. It has to be negotiated as locally as possible.

Reply on news.ycombinator.com