The White Hat's Dilemma
docs.google.com
docs.google.com
I didn't say anything. It was happening to "them", Chinese nationals. Not only that, but "they" should know better than to say sensitive things online, because even if we didn't install the back door, I reasoned, it wouldn't be too hard to get that data through various other means.
I really regret not only my participation, but not making a big stink about it. No-one did. I strongly suspect that that same system is being being used domestically, now. Clearly it was the wrong thing to do. I've regretted my role in that implementation for several years. I shouldn't have participated, and I should have protested. Even if it didn't stop it, at least the company leadership might have felt the heat. But I was a coward and I didn't want to lose my job, didn't want to fight a legal battle, and, like I said, it was just China spying on it's people, which everyone knew they do anyway.
And who knows? The news probably would have been ignored, or, if it wasn't, I might have been branded as a coward and a disloyal employee, betraying the people who put food on my table. And I being under 30, overpaid, over-priviledged, etc. I can hear the Fox News commentators even now. That, to me, has been the most difficult thing about Snowden, is that here's someone who did the right thing, who revealed wrong-doing on the part of our government, and there are a lot of people who say he's the wrongdoer, who attack him as disloyal and worse. A back door in a game used by China? Who would even care about that? And if they did, I'd just be torn to shreds, unemployable and with heaven-knows-what kind of future.
The reaction to Manning and Snowden, particularly the lack of strong public support, sends a strong signal that people don't want to know. They don't want to upset the apple cart. They don't want to challenge the government, they don't want to question it, not even when it's clearly violating it's own most important rules - the rules that, presumably, we've been fighting to promote these last 200 years. It seems hopeless.
If you are ever placed into a situation psychologically and ethically analogous to these experiments--a possibility that in a militarized and capitalist world it seems wise not to dismiss--and you want to acquit yourself better, what can you do? One option is to just have faith that you'll turn out to be an exceptionally good person. But that doesn't seem to me to be much wiser than assuming that no one with authority will ever encourage you to do something wrong. I think a better option, per the OP, is to think seriously in advance about what kind of situations might arise in your actual life and how you would like yourself to respond.
"If you were raised in the deep South during the 18th and 19th centuries, would you be aggressively racist, ambivalent or abolitionist in your views towards Black enslavement?"
The immediate gut feeling that people feel is important, because it doesn't end there. You then ask if that gut feeling can be reasoned to be consistent with their behavior, or if it's just a feel-good response. If someone really feels they'd be abolitionist, this should be at least somewhat reflected in small examples of day to day behavior. People don't reserve "goodness" for large deeds, they are usually the kind of people who help out in small, less noticeable/outspoken ways as well.
NB: This is not a test designed to prove how good someone is to another person, it's a self-assessment. It's asked to help the person learn more about themselves.
If you have an ethical choice to make, consider the reaction of your esteemed colleagues (get counsel if you feel you can), not that of the mainstream news. The general support for those such as Snowden and Manning among my peers would prove that indeed they are held in high regard among those to whom it matters. The decision you made was your decision and you should not curse yourself for it but learn from it and support those who did make decisions to release materials and information.
If Google really wants to make a statement about being not-evil, make Snowden a generous offer for employment while he's in Russia.
People like Snowden are not branded untouchable because people are ideologically opposed to his actions - that's a misrepresentation of what's actually going on. Whistleblowers become shunned because whoever they come into contact with will be elevated to accomplice status.
Google doesn't want that kind of fire under their ass. Altruism is all well and good, but there's a reason wh you're instructed to put your own oxygen mask on in a airplane emergency.
If a company like Google did that, they would immediately: 1. Place Snowden in United States lawful jurisdiction, 2. Endanger their own fiscal well-being, 3. Draw political attention and divisive sentiments to themselves.
I believe tech companies have a responsibility to take a stand, but being the hero and standing up to the man like that is essentially taking out Snowden and your company in a massive kamekaze attack that benefits only his detractors.
The most I would expect any company to do is donate matching funds to a legal defense.
First, thanks for coming clean. You're a human being who's made a really bad decision.
You should feel terrible for this. "They" are not merely your possible friends, or your relative's relatives, or your neighbors' cousins... "They" are other human beings who might someday decide to stand up for their human rights. "They" are millions of people you sold out for easy "dinner" & "rent".
And now the tool you helped build to invade their privacy is likely being turned on all your real friends, and your real relatives, and your real neighbors. And you're not even getting paid anymore.
You should feel terrible for selling out for so little. You weren't as afraid of being unemployed as you were of having to do something. You were mentally lazy and your moral compass was clearly defunct.
Too bad your behavior is not undoable and you'll have to live with it for the rest of your life. Maybe you can do something to recover for it in the future.
Hopefully your ethical failure can serve as an example of what not to do.
Edit: What about the employer's choices to go ahead with it? No anger directed that way? This guy should have fallen on his sword but his company gets a pass?
Please don't mistake my lack of attention to Blizzard's complicity as a 'pass'.
I actively campaign against folks who would bury their heads in the sand. In the past decade I have spent huge effort (preaching?) to convince my peers not to accept work in these Military Industrial jobs and to leave them if they're already there. I've had quite a bit of success in that regard, and I would encourage (as I tried in my response) that others would help educate their peers as well.
And I wish more people would stand up and say it to their friends and relatives when those people graduate college and start talking about working for the Military Industrial Complex, "Cyber" anything, or various other companies who build bombs, agents of war, or devices used to subjugate other humans.
I hope this person will work hard to redeem what he's wrought. Nothing else can redeem him, his work won't just disappear for his remorse.
>Too bad your behavior is not undoable and you'll have to live with it for the rest of your life. Maybe you can do something to recover for it in the future. Hopefully your ethical failure can serve as an example of what not to do.
I hope that others will send the message loudly and clearly to those that would create bomb parts or fragments of code necessary for snooping tools or facilitate those who would like to usurp peaceful human behavior.
You now say that the aim was to get repentance, when he's already stated he regrets it. At no point do you give advice about what he could/should do now to 'make up for his sins', apart from 'feel really bad'.
What do you want him to do to make up for it? Travel to China and tell the government to stop spying on their people using software he helped program?
My problem is with the comment that does nothing more than lambast someone without providing anything constructive. Offering some comment on how to approach such a situation next time, or what he could now, or projects he might think about contributing to, or anything else that might be useful to the person who originally posted or others reading the thread. As it stands, it's not much better than an ad-hominem.
The Internet is not a confessional and speaking your sins out loud here doesn't automatically absolve you. But that might be an impression one could get if the only responses were in the tone of "thank you for telling us, you're a brave and good man". So we need comments, like mr_spothawk's, that basically say "you're not a bad guy, but you fucked up; here's why and how exactly; let's hope we can all learn from this".
() he who has the gold, makes the rule.
I want people to wake up and realize that this is absolutely farcical. We still live in a democracy, and we still have the power to stop this behavior by 'the system'. It took generations of stupidity to get here and it will take generations to get us out, but, cynical as I am, I don't think all is lost.
It takes waking up and walking out of jobs that do this sort of work. It takes jury nullification [1]. It takes thoughtful decisions by graduating seniors looking for work. It takes political education and civil protest and HARD WORK, but it doesn't require violence between the states.
President Carter's quote: http://www.truth-out.org/opinion/item/17698-president-carter...
That whatever is legal or not can still go against your own feel of right or wrong. Who cares if something is legal somewhere and you don't agree with it. There are probably many examples of things legal in countries you visit which your moral compass would prevent you from taking part in. My personal annoyance being bullfighting in Spain (where I live 4 months of the year).
Sadly, it's only subtle because there are crazies around here and in the government who actually use that as an argument.
So... I don't believe in neutral ethical engineering decisions when the outcomes are crystal clear. I am not talking about ethical decisions on working at Zynga, at the end people can decide if they play their stupid games or not.
The world needs more people like you. We all do things we regret later in life. I have great respect for the fact that you've reflected on your involvement in this, come to regret it, and have now chosen to speak out about it.
I think this particular case actually deserves a HN story in itself.
Having lived there for nearly a decade, including middle and high school, I can tell you that people here just don't care anymore, and news papers and websites would know better than to publish a story like that anyway; you would have self-immolated for nothing.
In the interest of your safety and
the prosperity of the Chinese, all
your chat messages are henceforth
constantly monitored by government
officials. Have a nice day.
[OK]
But you might want to feel sorry about your government who clearly "improved" a lot in the last few years after it learned Yahoo's leak person email to Chinese government.
* I'm not a "white hat" or a "black hat"
* I'm not deliberately involved in any kind of "cyber" conflict
* I don't do what I do because I'm battling the forces of evil, or organized crime, or anything else
Instead: I do engineering. The same way a contract driver developer does, or a Rails dev. I happen to work in a particularly challenging problem domain. My work happens to have some interesting implications. But those implications are not the reason I work in the field; I work here because it allows me to grapple with compilers, number theory, low-level networking, hardware, OS kernels, and every imaginable development platform. It's about the craft.
I find this vantage point, which appears amoral, makes the ethical dilemmas easier to resolve. If a company like Narus asks me to help them make a network monitoring system harder to evade, I don't have to put that request into some ethical framework that considers the good that application might do. I just turn the work down. Same goes for the US Government; no, sorry, not interested.
Total respect for Alex (the "white hat consulting company" he founded is iSec Partners, our sister company and former archrival). I get the sense that Alex engages intentionally with these dilemmas, that he wants to be a part of something larger than himself and, I think, larger than the craft. As a result, sure, he has to live a carefully examined life, and make sure the projects he's working on aren't skewing his compass. I admire him for picking his way through those problems. But I'm every bit as engaged with the field as Alex is, and I'm here to tell you that you don't have to get tangled up in these kinds of ethical problems if you don't want to.
To call this position as "non-participation" or "categorical rejection" is intellectually dishonest, and to say this stance is some kind of blanket protection from future ethical considerations is borderline trolling.
If it had not been for the acquiescence of engineers who took part in the creation of PRISM, XKeyscore, etc. we... well, we would not have PRISM, XKeyscore, etc. Increasingly there is no such thing as an "amoral" position when it comes to a lot of these things -- you're either an entity who willingly chooses profit over principles, or you do something to defeat the evil as you see it (or, at least refuse to take part in it). In this day and age the conscientiousness of man is one of the last remaining defenses to fight the many evils, new or old, mercurial or familiar. It falls on all of us to think of the moral ramifications of our actions, in the workplace and off, and choose carefully and to the extent we comfortably can to see humanity continue prosperously.
I don't mean this to be a thoughtless, idealistic anti-NSA tirade, I'm frankly very okay with folks working on hip new technology that catches the bad guys, I just think your decision framework which is devoid of any ethical considerations is highly, highly dangerous and I wish for the good of us all that it doesn't catch on.
Oh, come on, man. If you think this is true in the fullest sense, you are not a thinking individual. Mere technological feasibility is 99% of the battle; implementation is the last, most inevitable step.
>the famous Edmund Burke quote: "All that is necessary for evil to triumph is for good men to do nothing".
Googler. The quote may be famous, but Burke is not. Like most men with noble fighting words, he had more noble words in him than he did noble fighting.
Inevitable? As in, happens all by itself? How is that thinking?
That's a pretty shit argument for being the one who folds. If everyone does refuse then it doesn't get built. If you refuse and they find someone else, that person may not be as good as you and may create a less effective surveillance apparatus which is easier for white hats to neutralize or dismantle.
Engineering has practical consequences. If you build something that gets democracy activists killed, you're the one who has to live with that. There are plenty of cool problems to solve that don't involve the construction of a surveillance state.
or you do something to defeat the evil as you see it (or, at least refuse to take part in it)
Refusing to take part doesn't absolve you of anything, according to your philosophy. That's exactly what Thomas says he does - refuses to take part. And by refusing to take part, you're still doing nothing to thwart what you see as evil.
> Refusing to take part doesn't absolve you of anything, according to your philosophy
I think in this context it's fair to interpret a refusal to work on something you deem evil not as inaction but as an act that makes it difficult for evil to prevail. If most good men did this they either would not find persons to complete the work or only be able to find persons who cannot do it well or do it completely.
To further clarify, what I am really saying is one's decisions at work which are detached from any ethical considerations is a problem, they're not just engineering problems -- they affect people, in good ways or bad. I hope everyone would make an earnest effort to determine the morality of tools, laws, policies, etc. they're in charge of creating or maintaining by accessing existing literature, discussing the moral considerations of their work with their peers and others, and then decide if they really want to be a part of that. And, as it happens, the chances are that since a lot of this stuff work requires high competency, whenever you find yourself in a situation where determining the morality of your work is exceedingly difficult, there is good chance you can easily find good work elsewhere that will give you the right engineering challenges without the difficult ethical questions.
You completely missed my point. You can't know how "evil" something is because "evil" is a point of view, not an objective fact.
> I think in this context it's fair to interpret a refusal to work on something you deem evil not as inaction but as an act that makes it difficult for evil to prevail. If most good men did this they either would not find persons to complete the work or only be able to find persons who cannot do it well or do it completely.
I think you would prefer that this were the case, but it's not. It assumes that everybody else in the world with the type of training required to do the task also turns it down. It also assumes that a young, idealistic programmer with a talent for [crypto/big data/whatever] isn't convinced he's helping to protect his fellow americans by taking the very job you turned down. In a nutshell, it assumes everybody has the same moral values you do, which is demonstrably not the case.
This brings up the question, if somebody is going to do that job anyway, is it enough that it's not you? In other words, is turning down the job enough to resolve your ethical dilemma? Personally, I choose Thomas's method of avoiding jobs that even make me think about it, especially when it comes to surveillance and privacy.
I don't know about the original commenter, but I fell into this position as being the best for me based on, "I am overwhelmed by the number of things that I need to know to make a judgement of good/bad over here, but don't or cannot know. There is too much random chance in my life to figure out how my actions play out. Until I grow wiser, let me do what that chance has laid my way, knowing fully well that I am operating in the dark."
The big, muddying parameters for me to answer 'is what I am doing right?' were:
1. In what context?
2. Over what timeframe?
The larger the context, the longer the timeframe, the more the number of competing principles I had to prioritize, often in inconsistent ways over different aspects of life. In the end, I defaulted to the original commenter's position, with the blind optimism that I would somehow, somewhere in the future get more clarity and wisdom through experience.
Okay, forgive me for getting even more meta here, but: You're operating under the assumption that I would doggedly stick to my misinterpretation (if it were the case), or maybe that I have some agenda to distort the messages in your comments? My comment history suggests otherwise: I'm more than happy to back off, apologize, and recant any misguided statements I make if it's pointed out rationally. Why doesn't the idea of calmly, non-abrasively trying to explain the breakdown in communication occur to you? Why are you so quick to elevate differences into an us-vs-them orientation on a personal level?
In fact, you too are part of something larger than yourself and as you work you make decisions that affect it (us) whether or not you think about it. Ignoring an ethical quandary isn't the same as escaping it.
So what ethical stance are you saying you take? You identify as "amoral" and seem to use that to mean "simply self-interested", where self-interest involves doing a craft you enjoy. But then you say there are jobs you wouldn't take for reasons external to the technology. cgag's question is a good one, and I think the contradiction there points to a flaw in the approach of starting out by thinking you can avoid ethical choices. Since you can't really, the only result will be that you make them without thinking them through.
I missed you at Defcon for multiple reasons, not the least being the opportunity to get your feedback on the talk as delivered. Maybe we can run a pan-NCC internal conference this fall and see what everybody else is working on. Chicago is nice and central between SF and Manchester.
A big part of the talk was my theory that our industry can no longer claim neutrality; like medicine or law, our actions have become innately entwined with ethical dilemmas that I feel to be better dealt with explicitly and ahead of the moment of decision. I don't think you necessarily disagree, since you lay out two lines you are not willing to cross even if you do not specify your reasoning.
I expect somebody as seasoned and experienced as you can make these decisions subconsciously without violating your basic principles. Younger, less experienced individuals may find this to be a greater challenge and they were the real target of my talk.
In my eyes your actions definitely make you a white hat, even if you avoid the label.
in the UK (engineering council)
"All professional engineers and technicians are bound by the Codes of Conduct of their professional engineering institutions. "
and in the US from the NSPE
Engineers, in the fulfillment of their professional duties, shall:
Hold paramount the safety, health, and welfare of the public. Perform services only in areas of their competence. Issue public statements only in an objective and truthful manner. Act for each employer or client as faithful agents or trustees. Avoid deceptive acts. Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession.
How would you hope other people would answer them?
This is, in a lot of cases, a nearly impossible obligation to completely fulfill, but in application, it leads to both a closer examination of privilege and to moral decisions and outcomes that are progressive.
Also if you live in US you should always put your own safety in the first place. US justice system becomes most significant threat to capable citizens.
It would perhaps to be more constructive to consider a positive model of integrity (Positive as in positive theory in economics). In many ways we have confused morality and ethics with integrity. Integrity when distinguished in the positive model it can be apply consistently across culture, societies, groups or organization (kind like the law of gravity).
For those who are interested, you download the short paper by Dr. Mike Jensen on social science research network related to positive model of integrity:
https://www.eff.org/deeplinks/2013/08/letter?utm_source=twit...
* the names are misspelled: first person is Sergey Aleynikov (not alinikov) and second person is Samarth Agarwal (not agrawal)
* in each circumstance, there was actual trade secret theft. That part is clear. The slide itself seems to suggest something beyond that, but they essentially took code that they wrote for their employer (and they signed contracts clearly saying that it belongs to the employers)
The point of that slide is that trade secret theft is a very old problem, and that there is a long history of criminal and civil case law to look to when punishing that kind of action. Those individuals were all charged under the Economic Espionage Act and face extreme penalties. I see this as another version of overcharging under the CFAA; the Federal Government has one standard for doing something on paper and a much harsher one for the same activities while using an SVN repo.
I am not defending the actions of those men, I just feel that the civil remedies that have been used for decades are more appropriate than having the soul-destroying power of the US DOJ turned against them on behalf of their employers.
The ethical dilemma exists for the technologist who performs the investigation and testifies against her former co-worker. What responsibility does she have to see justice done? I don't have an answer, but that was the question posed by the slides on justice.
But, I do recall I would get Mage (Honesty) if I did it, er... honestly. Today I get Bard (Compassion). I guess people really do change over the course of their life.
That's not rhetorical, I'm curious--it seems that virtually every company has some amount of proprietary information, isn't that what drives competitive advantage in software?
Recently I was offered a good and well-paying job. Late in the recruitment phase there was this far-reaching NDA I would have to sign. I asked to be given two days to read and evaluate it, which apparently labeled me as someone very strange, I should just flip through it in 10 seconds and sign like everybody else.
Two days later when I questioned their HR/Legal about some of the clauses they got very uncomfortable, and tried to avoid making a big deal out of them, saying it was "standard" and "at this site, they had never taken anyone to court based on it".
I didn't sign, and hence they could no longer offer me the job. Oh well, their loss, they would have to do without the value I could provide to their business.
Very rarely in the software side IME. Software is basically a commodity at this point - sure, there are better and worse ways to write it, but given the requirements and UI, any vaguely competent company could implement it in comparable timeframes. Competitive advantage tends to lie either in a faster turnaround time on making changes (but there's no secret to how you do that - hire good programmers and keep your codebase clean - taking a competitor's code is not going to help with this), or in the non-software side of the business (client relationships etc.)
Ask the opposite question, as an entrepreneur; why would you even ask someone to sign an NDA.