Google Workers Urge C.E.O. To Pull Out of Pentagon A.I. Project
nytimes.com
nytimes.com
"I can't think of a job or a career where the understanding of ethics is more important than engineering," Dr. Kearns continues. "Who designed the artificial aortic heart valve? An engineer did that. Who designed the gas chambers at Auschwitz? An engineer did that, too. One man was responsible for helping save tens of thousands of lives. Another man helped kill millions."
"Now, I don't know what any of you are going to end up doing in your lives," Dr. Kearns says, "but I can guarantee you that there will come a day when you have a decision to make. And it won't be as easy as deciding between a heart valve and a gas chamber."
To me this is incredibly valid for Silicon Valley engineers these days.
Sometimes the distinction is even more insidious. I did work on perpetual flight for drones, and Facebook had a perpetual flight project that had the goal of bringing internet access to remote locations in Africa. Sounds humanitarian, but I also didn't want to be responsible for subjecting poor Africans to what I consider Facebook's panopticon. Or maybe it would have been a net boon for the region? It's really hard to tell a priori, so the best I have managed for myself is just to try and stay in theory, where developments are further removed from direct consequence.
Separately, RE: bringing internet to new places, there's reasonable evidence to show that introducing broadband lifts GDP[1], so whatever you may think of Facebook, giving people high-speed connectivity to the whole Internet is probably a net positive. Disclaimer: I helped start Facebook's Connectivity Lab but no longer work there.
[1] http://pubdocs.worldbank.org/en/391452529895999/WDR16-BP-Exp...
No wonder I have such an alcohol problem...
In the case of Fritz Haber, it's the same man who saved billions from hunger and killed millions by prolonging World War-I. Excellent article on the same - https://medium.com/the-mission/the-tragedy-of-fritz-haber-th...
Sometimes, the decision isn't clear as black & white; especially when you mix nationalism & patriotism into it.
Mass intentional killing is wrong, pretty much everyone agrees with that.
Power, corruption and lies.
Those people often stand to personally (financially) benefit from unethical business decisions. Engineers have to think through all the dirty consequences, and still draw pretty much the same salary.
So, there are two aspects of that: first, do you trust the people you're designing a weapon for to use it for justifiable purposes, and second, how does the weapon itself influence whether it can be used for just or unjust reasons? Gas chambers, for example, can only be used to kill someone whom you are already able to coerce into entering it; its only use is to murder a helpless prisoner. A Spitfire or Hurricane can shoot down bombers but not necessarily do that much damage on the ground. Orwell had an interesting perspective on this question: "Thus, for example, tanks, battleships and bombing planes are inherently tyrannical weapons, while rifles, muskets, long-bows, and hand-grenades are inherently democratic weapons. A complex weapon makes the strong stronger, while a simple weapon — so long as there is no answer to it — gives claws to the weak."
I really don't have answers. The stance of, "weapons equals bad, therefore it's wrong to ever help design weapons" is a pacifist notion, and maybe that appeals to you, but if you're not a pacifist, it becomes a complicated question with certain political ramifications.
If one could engineer a solution to save lives, but instead uses his/her talents to build the nth iteration of, say, a food delivery app, does he/she bear any ethical culpability?
Because it seems the bigger challenge for engineers is not avoiding overt evil, but in the opportunity cost of spending their time on work of significantly lesser consequence than their talents might allow.
https://www.asce.org/code-of-ethics/
Yes, yes, we have a responsibility to not defraud people, and to not work on obviously evil projects, and even to speak out if your employer is, say, building a bridge that is not up to the clearly defined, established standards. (which is super different from when you think a bridge will collapse, but can't explain why in a way that other people can understand. This parenthetical part is particularly important, I think.)
But we are free humans, and if we want to waste our lives on trivial things, so long as those trivial things don't hurt others, that's our choice. We are humans, and humans need entertainment, we need play.
http://ccr.sigcomm.org/archive/1995/jan95/ccr-9501-clark.pdf Here they say that the design of the internet was very much determined by the military priorities of the project (Telcos had different priorities so they came up with OSI where the link layer is supposed to be reliable, as opposed to DARPANET/IP, where the link layer/IP routing is dumb and the endpoint is smart; this allowed link layer to be pluggable and IP routers to be cheap)
I used to find myself most of the times confused while using technologies whose CEOs, mission I do not agree with. It felt as if I was contradicting myself. But then I realized the dialectical nature of this relationship which is if you want to create something better, improve something it is only natural that you will do so using the existing tools available in your world.
A great example is Facebook. There are things you might not agree with, you might hate it but if you look closer you will see volunteer groups to political campaigns to marches and protests which are organized only via Facebook. One of such political activities one day could as well change Facebook or the world we live in drastically.
You can apply this to other cases. You might have an anti-capitalist philosophy podcast hosted on Amazon or Spotify or Apple which in essence contradicts the very existence of such companies. But in the end you are using them to deliver your message, cause change etc.
I am curious about what others think about this and how they feel about looking at it from a dialectical point of view as I am continuously thinking about this and still forming my thoughts.
But there is space for ethical contemplation during an arms race.
No one hates war more than soldiers.
That's the truth.
Did the engineers of the atomic bomb save millions of lives and usher in an era of unprecedented peace? Or did they slaughter millions of innocents and sow the seeds of humanity's destruction?
It's been 70 years; we've had plenty of time to make up our minds.
Many inventions are not so easily classified as innately good or evil.
>Kearns was a member of the Office of Strategic Services, the forerunner of the U.S. CIA, during World War II.[1][2][3]
Somehow I don't think that Kearns would have had an issue with wriing software to analyze video footage for the military.
Merely offering a force projecting capability is not evil.
If you look at world history, the only concrete definition of who has most political power is this: it's the party with the most capability for projecting violence in a given area.
Small close knit groups that live outside of large populations can live independently and do without military force. Once you have a large population with more or less fixed hearth and home the party with most violence projection capability owns them politically.
This dynamic is played in almost ridiculously minuscule scale in the peloponnesian wars of the greek city states of antiquity.
The pathological manifestation of this principle can be viewed, say, in the rise of ISIS and the Somali warlords.
While a modern state is seldom a benevolent actor, at least in the western countries it's the best of known alternatives.
In this framework violence is a key tool of the state, just like good governance, tax collection, etc.
Given that violence is a necessity, in my opinion designing military instruments in itself is not evil. It's not as good as designing new vaccines, so there is some ethical scale in the matter, but I'm damned if I can put it in concrete terms.
Are you sure it's a good idea to become less influential on the world stage?
What gets me now, is how much the US economy and focus is on death and violence. Offensive death and violence. We're bombing people all over the Middle East. And have been for years. Tangentially, I find 'March for our Lives' ironic given the amount of, to me, unjustified, violence and death the US country exports to children and families around the world.
For self-defense only? In an ideal world, yea, invent the meanest things possible. That philosophy still is compatible with a strong military. But then again you have Gulf of Tonkin, etc., so who knows!?
Lately I've been telling myself this whole wargame is a game I'll never be privy to what's actually happening.
I think people often have that twisted. A lot of the U.S.'s expansion is cultural, not military. That is to say: people liked what they were doing, and became a fan. Think: music, movies, and to some extent, literature, and now you might add software and consumer electronics to that list.
And even when looking at Rome, can you imagine how much harder those repeated annexations would have been if the Roman culture wasn't respected?
I'd say that being capable at least of defending oneself, independently, from most individual powers and believable coalitions, creates a certain amount of stability.
But, there is power in organizing, or unionizing, for collective bargaining.
Well, given how unimaginative a leap the national socialists took from the national fumigation program to the gas chambers, I doubt there was much more to it than "take bug poison room tech, use it on humans".
There's no such thing as an "engineer" in the real world, and there's almost always somebody more unscrupulous, devout, or patriotic than you who knows or is willing to learn what it takes to finish the job, and school is not going to get in the way.
Added: Engineering is a practice, not an immutable identity. Even if you publicly shame all the "engineers" away from contributing to an effort you disagree with, replacing them is a matter of learning enough of the practice to complete the task. Even if we were just talking about what is in paper books today, it's hard to say that anyone could have control over who is or isn't capable of engineering efforts.
If you believe that doing something is wrong, the idea that someone more unscrupulous will do it if you do not must not be used as an excuse for doing that thing yourself. Giving in to this notion will prove yourself to be the one who is unscrupulous.
Just recently in the AI space:
- Siri was spun out of a Pentagon project -- look up SRI International and CALO. Its purpose: a "soldier’s servant".[1]
- Autonomous driving is a direct evolution of Pentagon-funded efforts -- see DARPA Grand Challenge.
And it's not just funding the early research, it's procurement like this too. Military procurement has also supported the development of technologies when the commercial market couldn't.
Again, I support the employees and hate the fact that in order to develop medical lasers we first have to figure out how to shoot down missiles with them. It's hugely inefficient, could spell our doom, and if you think about it, fundamentally undemocratic. (Gives elites more power to direct taxpayer dollars under the rubric of defense.) But this should be a basic part of any story on how Silicon Valley works.
And to think that SV has a large population of supposedly "small government" Libertarian Capitalists... oh, the mental gymnastics in that.
[1] https://www.amazon.com/Dream-Machine-Licklider-Revolution-Co...
This is a national budget problem. If you're in the tech world long enough, it becomes pretty clear that the only way to get Big funding is through military affiliation.
This puts a huge selective bias on what kind of technology projects actually get big funding, and further it prevents the benefits of those projects from reaching the community for years, because the military overlords demand secrecy sole use of the technology until it gets superceded.
We need to cut a huge chunk out of the military budget and give it directly to the tech sector, so that big innovative projects are actually possible without having to be military.
Lookup "the secret history of silicon valley". There was no tech world in the bay and massive funding for radar research post-WW2 bootstrapped what is now silicon valley.
It's not a national budget problem, it's just the history of why things happened in SV.
MIT's Lincoln Laboratory made 27% of their revenue (roughly a billion dollars) in 2016. That's one research center.
I believe the silicon research/production that started Silicon Valley was for military needs. Military paid for it.
First Computer of US and Britain? It was funded and created during war for military purposes.
And what about China and Russia? Will THEY stop AI research just because Google employees demand it?
Innovation doesn't need defense support, innovation will come anyway. We just choose to allocate much of our resources to the military so that's where it gets spent. Later we can claim it as a win for defense. A happy surprise benefit of trying to kill each other.
All you need is people, time and money. Profit.
Many years ago I choose not to work on a local defense project. I found out that the project had ties to Chinese defense. I'm fine with my choice because I find their leadership somewhat oppressive.
Another time I saw Iranians on the campus. Another awesome opportunity for interesting work that some might not want to get involved in.
I know someone who did work for Mugabe. Hey, cool project. It's easy when you justify it.
I've called SV the military's generalized R&D department.
However, if you think an ideology is has logical errors and requires mental gymnastics, you likely don't understand it well enough to intelligently criticize it.
That's true, but not really in the general way you argue. Military spending on bay area R&D has been more or less insignificant for the entirety of the internet era. It exists, sure, but it's not driving meaningful revenue for any of the big players. Silicon Valley since the mid 90's has been a consumer thing only.
And even in the genesis of the valley, it was just one serendipitous application (missile guidance systems, which were willing to pay huge sums for early transistors which were literally 100x lighter than vacuum tubes).
Not only that, but when Youtube and Google block content, censor people, or otherwise prevent free speech, the common rejoinder is "they are private companies, they don't have to abide by the 1st amendment". Any company that is received taxpayer money, whether through direct subsidies, grants, partnerership, or any other avenue, ceases to be a "private company". I believe strongly that if you are truly private, and you operate entirely with your own private funds, and wholly-owned, privately purchased infrastructure, you are free to say and do what you want - silence any voice or opinion that you don't like. But once you receive public funds, in any context, along with that comes with the responsibility to the public. Despite being "the way it works" currently, this system is incredibly corrupt and logically inconsistent. In a world where the Constitution was respected, enforced and held inviolate, a company that received public funds (like Google) would not be allowed to declare themselves a private company with total autonomy on one hand, and grab countless millions in taxpayer money in the other.
The military application in question is legal and is approved by a duly elected government that supports it politically. In earlier days, employees generally would see this as just doing their jobs in developing technology that their employers wanted developed and would not concern themselves about ultimate uses and applications. In other words, doing your job is personal and, as long as you do it honestly and work hard, you should not be faulted for doing it as requested by your employer. That was always the standard. What then is the new element from which this sort of employee-driven demand arises? Is it morality? In other words, if I help develop A.I. that can be used for all sorts of things, one of which happens to be military-related, is the effort "evil" if the employer for whom I develop it agrees contractually to provide it to the government for a wartime/military use that can kill people? Do I really make a difference for the good if I convince my employer not to do this if all this means is that the company down the street gets the contract and the military gets the same results, albeit from a different vendor? If this is so, then I assume that you as an employee can make no practical difference in making the world better by insisting that your employer forego this particular form of contracting opportunity. If you succeed, your employer misses an opportunity but the evil you see being released into the world still gets released. It just means that you do not personally contribute to the development effort by which it is made possible.
Of course, it might theoretically be possible to persuade all persons working in the field of A.I. to ban further work that directly helps the military. But that would seem a practical impossibility. Many people in all countries believe that military technology of all kinds is proper, legal, and politically supportable for purposes of self-defense or for some other overriding purpose they deem proper. And certainly, there are bad people throughout the world who are eager to use any technology that comes their way for overtly evil purposes such as misuse of an atomic bomb. Unless and until human nature is fundamentally transformed, that will never change.
So, what is the answer in a country such as the United States where people and companies have the freedom to develop A.I. for any lawful purpose and where some inevitably will do so for a military purpose of which you disapprove?
You are then left only with a political solution: use political means to gain control of the government and the military and apply the force of law to ban the military use of which you disapprove.
So this is either a personal act of futility by the Google employees or it is a case by which they cannot separate the personal from the political and thereby insist that their employer sacrifice particular economic opportunities to ensure that your personal actions do not support a political outcome of which you disapprove.
Even then, does this mean that your employer should cease working on A.I. altogether? For, just as cash is fungible, so too is technology. Every improvement you make in A.I. might have an immediate use of x for your employer but, as humans collectively do this for all sorts of improvements, the results are there for the taking in the future for military applications of all kinds. In other words, you cannot put your improvement in a box or control it so as to limit its future uses (at least not in a free society). The computing technology of recent decades undoubtedly has bettered many aspects of life but it has also greatly magnified the lethality and utility of military applications so as to make the world far less safe. And this was inevitable unless a supervening agency were to have used forcible and totalitarian means to suppress such technological development from inception. Since no such supervening agency existed or even can exist in a free society, does this mean that all engineers and technical developers have blood on their hands because, ultimately, things they have done were used for applications of which they disapproved? Of course it does not. Nor would people today working on A.I. be held morally or legally responsible for ultimate downstream uses made of their work of which they would not morally approve today.
But this brings us back full circle. In the long run, you cannot stop such uses (or misuses) made from your technical development work. Nor can you be held responsible for them even though you contributed to them in some remote degree through your work efforts. Why then should it make a difference if your direct work efforts for a company like Google are applied to a military application of which you do not approve but which is legal, politically approved by the governing authorities, and will happen anyway regardless of whether Google is involved?
The puritans of old tried controlling the morality of others by shunning and shaming and doing it to an extreme degree. They failed miserably in their efforts because humanity is what it is and followed its own course without regard to external religious constraints.
This sort of effort by Google employees is obviously different in that it is not religiously driven but does it amount to anything more than a shunning-and-shaming method for trying to impose one's sense of morality on others by signaling that this way lies righteousness and everywhere else lies evil?
If this is what "don't be evil" now means, then Google will need lots of help going forward because every cause under the sun can be used in the same way to shun and shame. We then have management by a corporate board as may be swayed to and fro by any organized protest of the moment.
Whatever this is, and however it might be defensible in "sending a message" or whatever, it is a sure way to put a company at a competitive disadvantage while accomplishing nothing practically. It may further political goals but, if those are the goals, better just to try to advance them directly and not by attempting to shun and shame your employer (and your co-workers who may disagree with you) into submission. The personal need not be political. If it does become that way, a new form of puritanism will hold full sway to the detriment of all.
"It's hard to define things, so why bother at all?" ... "Attempting to circumscribe the affects of your work is difficult, so why have a moral stance at all?" ... dancing through loaded assumptions like "duly elected" and "democracy", finally concluding with a tired crescendo of capitalist "competitive disadvantage".
I would merely counter: if we are the future, then we can't all be lazy sods, especially those of us empowering the greatest systems, information and power structures on the planet. Give a damn, it's your moral duty. Intelligent people recognize this.
This is a false history. In the 1960s the U.S. was full of young people questioning whether they should just do as they were told and be a good, dutiful employee.
Since then there has been a massive campaign to roll back what was called “Vietnam syndrome” — the idea that you should consider the morality of your actions and contributions to society, not mere legality. Hence all the passionate Hollywood WWII dramatizations, the Greatest Generation, etc., portraying war as a tough but noble effort in which we must all unquestioningly sacrifice for the greater good — de-emphasizing much of the horrendous atrocities that have been perpetrated by the U.S. military in Vietnam, Iraq, support for murderous dictatorships in Central America, Indonesia, and so on.
Noam Chomsky has written extensively on this. One essay in particular is called The Backroom Boys — a reference to the chemical engineers at Dow who developed napalm within the relative peace and equanimity of a laboratory.
> Whatever this is, and however it might be defensible in "sending a message" or whatever, it is a sure way to put a company at a competitive disadvantage while accomplishing nothing practically… If it does become that way, a new form of puritanism will hold full sway to the detriment of all.
Whatever your opinion of the anti-war activists of the 1960s, puritans they were not.
Google has users in almost all countries, and even our friends in other liberal democracies do not see the U.S. military the same way we do. Perhaps some Google users' family members have even been killed by the U.S. military. This presents a perfectly reasonable business reason (one that has nothing to do with "the personal, the moral, the legal, and the political") for Google to turn down AI drone contracts.
We are starting to see the shape of the AI driven future and it is not pretty. Autonomous drones and other robots and surveillance patrolling systems establishing strict inescapable authoritarian control.
The elite employees at the global AI leader are best suited to see the coming dangers and are sounding the alarm bells. The outlook is bleak but moral engineers are going to be one line of defense in this fight. And I hope they keep doing it.
There's a famous quote for this:
It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter.
https://en.wikiquote.org/wiki/Nathaniel_Borenstein
Of which I believe the meaning is yes, it's evil. It's handing a toddler a loaded gun sort of evil. If you DestroyBaghdad, you've limited the harm your program can do to what is specifically required by the situation. DestroyCity is easily misused in the wrong hands and should be carefully considered by ethical programmers.
Doctors solve this by disallowing unethical members of their profession to legally practice. Programmers should consider becoming an ethical profession, because depending on others in the field to do the right thing and police themselves hasn't been working out.
Do you not believe that society is only the sum of its parts? Do you not believe that the mathematics of society can be changed, the more parts of the equation object to letting their talents be used for unscrupulous goals?
I would point you towards any cultural shift in modern society, and how it began—usually, as the imbalance of classes further divides, until one class can't tolerate it any further, and uses what power they have to reset the scales. That is what is happening today, and it isn't a fluke of the short attention span of the beast of humanity. It is a conscious, concerted effort of people in this country who are tired of existing in a system devoid of morals. And to frame it as something like embracing the status quo, or becoming a puritanical society, is simply a false dichotomy.
You might want to read some history. E.g. history of the development of nuclear weapons.
This sounds remarkibly similar to the "superior orders" defense given at Nuremberg [1]
(Those who gave it were hung from the neck until dead.)
But I also wonder, if your employer asked you to provide some legal cover for something you found unconscionable -- like maybe draw up incorporation documents for organizing sex tourism to a place where it's both legal and likely to involve slavery -- would you subscribe to the same argument about how you should just do your job and not ask questions, how someone else will do it if you don't, and dutifully provide the legal services?
And if you think that hypothetical scenario is meaningfully different from this one, could you describe how? (I don't mean to try to back you into a rhetorical corner -- I'm genuinely interested in your response.)
This is basically the "just following orders" defense. Your argument rationalizes doing nothing.
As a moral person I think you basically have two choices: either don't work on things you think can be used for evil or if you do work on those things step up and make sure they are used responsibly.
You never know what will be the initial trigger for change, who will be inspired or whatever. Take this guy: http://www.ecns.cn/2018/01-02/286632.shtml Why should some basketball celebrity and his campaign have so much effect?
Google supporting American military may have negative effects on Google the corporation, especially considering that the rest of the world is a bigger economy than the US.
Something I would add is that a lot of people don't understand how fundamental military R&D is to the collective progression of knowledge and technology. Take almost any common technology that we use today (computers, gps, rockets, airplanes, cell phones, radios, the internet, etc) and you will find it came from military R&D and use in war.
Since AI and all its related parts are the new technological hotness, to put it mildly, it only makes sense that Google, one of the companies on the forefront of this technology, would work with the government/military to do research and find ways to apply it with their scope.
You made a point of talking about cryptography. The US government also classified crypto as munitions in order to control its usage and export: https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
This had very negative effects on cryptography in general (see the FREAK exploit: https://en.wikipedia.org/wiki/FREAK )
Also, why wouldn't this be employed to better monitor Stingray (https://en.wikipedia.org/wiki/Stingray_phone_tracker) systems?
I don't think the government (and specifically, the military) should be viewed as a non-partisan actor when it comes to technology.
This is my perspective, because I read too much scifi: I can conceive of a couple possibilities in 300 years.
1. Humanity is gone or stone-aged. Either because it failed to colonize before being wiped out by disease, because it nuked itself, or because it implemented AI in a way that got itself killed.
2. Humanity has turned peaceful, formed a global community (if not wholly, then at least nearly so regarding scientific resources), and colonized.
I draw such a sharp line because I don't see how the resources to colonize can be mustered without justifying it with an arms race unless peaceful cooperation is established. Without this, I think somebody is going to put a nuke in orbit around Mars and call themselves King/Queen of the inner solar system.
Therefore, I think we should be working towards the cultural changes that I believe necessary immediately. This is why I'm a staunch proponent of universal education, universal healthcare, gun control, etc. I think there's no better time than the present to "be in the race together" globally than now. Given climate change, death of bees, super-bacterias, and narcissists with penis-size complexes holding fingers over red buttons, I see a ticking clock.
Avoid (1) by tackling culture, is my theory, and I think that's what these googler employees are here. (I post this very much looking forward to being challenged on all points)
$700,000,000,000
Or as Obama said in the 2016 State of the Union:
"We spend more on defense than the next 8 nations COMBINED."
Keep in mind, most of that 8 are allies.
I struggle to see why we should be praising a self-fulfilling prophecy.
For the purposes of being better at waging war. I am not disagreeing with your statement, but I think leaving out the reason why is treating the military like it's "NASA with a few guns laying around" and implying that one can work on weapons systems without having any moral culpability for how they are used.
Possible reasons, using Wernher von Braun's rocket development as an example:
- He, or somebody else, would have invented rockets anyway at some point
- That 'some point' would have been much earlier if the money spent on military (research & some of the rest) would instead have been put into rockets in the first place
- even if the two mechanisms abovc were not true: I'd probably be willing to forgo the benefits of the space age if it meant undoing WW2 and the Holocaust
- In most ethical frameworks except utilitarianism, it's strongly discourage to accept some evil for some perceived greater good. Meaning: Even if rockets have saved far more lives than WW2 vanquished, participating in the project is morally dubious at best
Specifically to Google's project:
- This seems to actually invert the usual idea of the military funding basic research that later gets adapted for civilian use. They are building on the fundamental research into AI already done by Google and others, and using it very specifically to kill people. Any advances useful for other purposes would seem to be even more coincidental than usual.
- While "minimising collateral damage" i. e. civilian casualties may seem to be a worthy cause, it cannot be ignored that such advances are likely to result in greater use of the technology. Drones are actually the best example of this effect. Just look at the Obama administration's willingness to expand the use of drone strikes, which was a direct result of the technology appearing to be the lesser evil compared to manned missions on the ground.
> DoD/Pentagon involvement in artificial intelligence research shouldn't be viewed as a necessarily bad thing.
Let's be spesific: they are using current AI algorithms to track everyone's movements from airborn imagery. It will be used to kill people in drone strikes and for population control.
Another thing is to specifically choose to dedicate resources to one of those areas instead of another area. Why can't google bid on AI projects for better detection of geological landmarks in other planets? (which I'm guessing is a similar endeavor to what the article mentions - and already has ongoing research)
That's a direct application of tech to killing people, not some abstract network or compiler tech. There's a big difference.
This also has the benefit of getting war averse liberals to say nice things about the large standing army and bloody foreign interventions of the US empire, most of which have failed to produce anything other than blowback and civil war across the globe.
https://archive.org/details/NoamChomsky-ScienceMilitaryFundi...
- The Global Assassination Grid: The Infrastructure and People behind Drone Killings: https://media.ccc.de/v/33c3-8425-the_global_assassination_gr...
- The Drone Papers: https://theintercept.com/drone-papers/
- Graphs, Drones & Phones: The role of social-graphs for Drones in the War on Terror: https://media.ccc.de/v/32c3-7259-graphs_drones_phones
Also, many people, including myself, consider drone strikes to be state-sponsored terrorism. Just because some part of the military may be justifiable, does not mean all of it is.
The Google workers urging their CEO to pull out of Pentagon projects are the ones who do not have such short memories as to be distracted from the blatant deceptions and disasters carried out in recent history, with no sign of relenting.
What the google employees need to do is push for a civilian government research and science funding effort, like ARPA was back in the day. Unfortunately that sort of funding is politically dead, while military funding is always in style.
Reminder, if you work in tech you are collecting blood money because everything we use was initially funded by the military for military application. If you are going to attack Google for this and don't leave the industry you are a hypocrite
And what do we have to show for it? Wikipedia is great and Youtube is entertaining, but the rest of the internet is horrible. I say that as a software developer.
Due to the wonderful internet our children now have attention spans far below the time a typical class lesson takes. The smartest brains on Earth are not building spaceships or curing disease, but rather implementing systems to get you to click on ads. Our bodies and habits are being tracked, our opinions manipulated, and our privacy is being handed of willingfully for a chat and photo-sharing platform.
Military tech _often_ has implications far beyond what the original scientists and engineers intended. Sometimes for the best. But as you allude AI to the internet, don't forget how much damage the internet is doing to us even as we enjoy it and profit from it.
I think DoD/Pentagon put up the funding for what later became Google maps. Not to mention the GPS constellation we all enjoy using.
Its a shame progress has to be made through these channels but it has proved very effective in the past.
We need weapons to fight an enemy like this, with help from US technology companies.
also it is not surprising that these things are invented by the dod when you're spending ~600-700 bilion a year on warfare and intelligence operations.
That last sentence of yours rubs me kinda the wrong way; that is a dangerious premise that could set us back ~30 years when the cold war and the arms race was well and alive, but this time we'd have robots and AI
The military-industrial complex may have become far bigger and perhaps in some ways a burden, but the world is a dangerous place and becoming more so. In my view, there is a perfectly valid argument that working with the military is the right thing to do.
At the same time if individuals have pacifist leanings and do not want to work for the military, i would hope corporations respect that.
Are you basing this statement on a general feeling or on a specific data-based metric? The military industrial complex greatly benefits from this common sentiment, but by most all quantitative measurements the world continues growing slowly safer, average lifespans increase, etc. While years-long aberrations exist (e.g. civil wars), the overall trend is very clearly antithetical to your statement.
What are you advocating? Arming governments with autonomous killer drones in order to "protect democracy"? Because the world is a "dangerous place"? I don't even...
Nope. Not even in the slightest.
https://en.wikipedia.org/wiki/File:Waves_of_democracy.png
https://en.wikipedia.org/wiki/History_of_democracy#Waves_of_...
Pacifism isn't viable when some of the biggest countries in the world are run by totalitarian regimes. Objections to participating in the military industrial complex are a luxury of countries verging on global irrelevance. The proclivities of the West to engage in self-flagellation over these issues are well known to and exploited by less liberal regimes. If only it were not so, but I don't know what the alternative is, just as the physicists in the Manhattan project saw no alternative.
I'm not even sure what the consequence of this argument is; pretty much anything you build can indirectly contribute to the military industrial complex, even something innocuous like dev infra. But I also don't think that weak "everything is equivalent" argument means you're suddenly absolved of responsibility. One thing I am pretty sure of is that it must feel awful to waste your short time on earth on building tools specifically for killing.
The fact that the military funds research doesn't change that.
After all, I would not be here, and my job would not exist, and the freedom I have to do this work would not exist, if soldiers like my grandfather hadn't stepped up to Hitler and Stalin and Mao and say "No, you're wrong, and we're willing to use violence to protect our way of life". Just my 2c but I definitely respect the ultimate sacrifice that my grandparents generation gave to create this quite nifty Liberty bubble that these few generations get to play in.
Keeping "free" nations safe and prosperous is as fine a goal as any.
Boy they pay great though...
These smug generalizations, if you did not vote for X, you are racist! If you are not for A, you are for Genocide! are not going to win you over people that are not already in your camp.
You come across as wanting that sweet sv $ and benefits without having to make any moral choices - if you got to work for CNI company's you know what the deal is.
And before you ask yes I did face that at my first job when I analysed some high speed film from one experiment and though oh I know what that "objct" is and my office mates said yes that's probably REDACTED.
I don't agree with this. Many open source projects have been used for sinister means (not just by governments). But does that make those contributors complicit? Should those contributors be safe-guarding against all possible unintended use cases?
It seems like you're painting all military efforts and funding in evil terms. But these efforts are directed by many different people, each with their own goals and agendas. Some of those efforts are intended for the betterment of humanity, while other efforts are at the detriment of humanity (at least in my opinion).
I worked for several years at a large defense contractor. The majority of our funding came from military contracts. I worked on a project intended to help detect and disarm explosive devices. And I worked next to a team that was trying to rapidly detect infectious diseases. We all thought we were doing work that would benefit society. But virtually any technology can be used for malicious purposes. Does that make us complicit if those technologies are ever used in malicious ways? Do the evil actions of some people in an organization make the entire organization evil? Does that mean that all military efforts are unjust?
Technology is neutral - who it's used by and how is not.
I don't care what you build, if it has the goal of improving a workflow or automating a process then it will de-facto support one organization over another.
On the other hand, this bothers me a bit because it continues to allow people in the valley to maintain a (sorry to use this word) delusion that what they are doing is "moral". If Google stops working with the Pentagon after this petition, people in the valley will pat their backs and enjoy how they are making the world a better place. They will not have any incentive to rethink the sale of user data to advertisers, creating highly addictive mentally harmful products, etc.
Overall, it's good that at least people in the valley are somewhat mindful of their actions and care about society (compared my current industry, finance). I hope they can be successful at a deeper level.
People refusing to work on military technology isn't "appearing moral". It's just "moral", at least if you consider war to be generally amoral.
People here and elsewhere have been throwing the term "virtue signalling" around a lot in the last year or so. This is a perfect counterexample: Google employees speaking out against military tech are shouldering the risk of appearing disloyal to their employer. If Google were to cut their military ties, they would forgo whatever benefit they previously sought in that relationship.
I don’t really get why anyone would want drones to be less accurate, but maybe that just means I’m just a no-good warmongering fool, right?
> They will not have any incentive to rethink the sale of user data to advertisers
Is it not possible that one will lead to the other? We're seeing the beginnings of a real examination of the power Silicon Valley giants have. Recent revelations about Facebook, amongst others, shows that awareness of advertising profiles etc. is increasing as well.
So while it's great these protestors want to protect us from AI, those in China, Russia, India, Iran, will endure a greater threat from AI and it behooves us to be at the forefront lest we fall behind.
In other words a moratorium can only work if all participants agree and participate. Sort of like the anti nuclear proliferation treaty.
Note : I do not agree with the petition, but the question was too tempting.
More seriously: the US has nuclear weapons, rendering them safe from any direct confrontation with enemies challenging them technologically.
They have a conventional military 10x the size needed for the sort of intervention that actually succeeded in the past (the Balkans, the first Gulf war). The rather unsuccessful wars (Afghanistan, Iraq) were lost not for technological reasons or any other dimension of military might, but because the US military sucks at convincing people of the benefits of democracy.
Want to win the next ten wars? Put the money into Hollywood blockbusters and the world's best Halal fast food chain.
A lot of Silicon Valley's justification for its immoral actions rests on the _belief_ of the inevitability of outcomes. It's ok to monopolize and centralize everything, discarding the lives of people who are negatively impacted because it's inevitable. It's ok to make robots able to kill people better because it's inevitable, etc.
The thing is, it's a _belief_ that this is inevitable — not a proven fact of nature. And people who don't believe in it will logically withdraw their cooperation from building that future.
Both sides are 'proving' their point of view. If enough people object, then it wasn't inevitable after all. If enough people work to bring us into that future, maybe it was. But thats the struggle and moral choice we face.
A) not be developed without support from Google, which is categorically untrue; The DoD might simply find it more difficult to acquire top talent if that talent is opposed to the work on philosophical grounds.
And B) be necessary for the US military to remain the most advanced and/or powerful in 20 years, which is also exceptionally doubtful - unless you believe ML to have such effective wartime applications as to render a nearly threefold larger military budget (compared to China, the next largest) moot.
Nukes are enough to stop any war from happening between any of the most powerful countries in the world.
What else do we even need?
The various factions in the various civil wars around the world can kill themselves in their own war, and we shouldn't be a part of it killing even more people.
That moment when you realize your mega-corp employer's "culture" is merely just another tool to get your to do their bidding.
How do the share holders feel? I'm sure they're betting that almost every employee will stay no matter how their technology is used.
To put a number on it: I am absolutely certain that Facebook would be willing right now to sacrifice 25% of their revenue for a bump in their reputation (among developers only) that puts them at Google's level.
(In the same vein, Google's NetPAC has been quietly donating the max to Republicans like Paul Ryan and Steve Scalise, while most of their engineers openly oppose them.)
And you want to make the program "more efficient."
This is the gulf which I cannot seem to bridge.
Extra-judicial killings of "terrorists" in "bad places" using flying killer robots is just batshit insane to me. I cannot for the life of me understand how that conversation would go with, e.g. Thomas Jefferson or Alexander Hamilton. Or really anyone who believes in human rights.
Then “AI” should be the least of your worries. Not everything is a software problem.
Their only intention here is to scale up their drone programs.
If, as an engineer, it is against your moral code to do any work that supports the military, your choices are limited to working in small companies where everyone is focused on the commercial products and services you are delivering. And even then, as some games companies found out, the CEO might do some collaborative work with the military for training or something.
It should come as no surprise that Google teams up with the Federal government on things.
This is nonsense, Google has plenty of revenue and other non-defense avenues for growth.
Companies are not beholden to chase every last dollar, especially if it could alienate their workforce.
I don't have the same anti-defense leanings as others around here, but to say Google has no choice is ridiculous.
Google probably couldn't (and shouldn't IMO) stop the DoD using Android phones if they want to, but they can certainly refuse to work with the defense sector.
Perhaps not, but that doesn't necessarily imply that Google can't pick and choose what technology it sells or provides to the military. They can, for example, serve the administrative needs of the military (e.g. Google Apps), but not necessarily provide technology that directly improves their weaponry.
Asking the business to reconsider before leaving is legitimate.
And their excuse that "they're only analyzing images" is a joke. Analyzing the images and "identifying objects" (their words) is probably 95-99% of the job in a drone strike. So if they're trying to make people think that their role is minimal in drone strikes, they're failing hard at that.
I also think Eric Schimidt, who until last December was both Alphabet Chairman and working for the Pentagon, played a big role in this. Now he's still a "technical advisor" but if he continues working for the Pentagon I'd prefer he has no official affiliation with Alphabet.
It seems Schmidt would like Google/Alphabet and Pentagon to have a much deeper relationship, if you can read between the lines in this post:
http://www.defenseone.com/technology/2017/01/pentagon-needs-...
Take, for example, RADAR. Should those that developed RADAR, during or just prior to WWII, and abstained because of its possible wartime use? Wikipedia says "In the 1934–1939 period, eight nations developed independently, and in great secrecy, systems of this type"[1], so those that were left out faced a real danger in the coming years if they didn't have something to compete. Yet, RADAR has been used to great effect in many weapon systems and by many countries since WWII in ways that that most would agree are horrible. Then again, it's made commercial air travel much safer than we could have hoped to achieve at this scale without RADAR.
It's a complex situation, and I think it deserves a complex discussion. Distilling it to "working with the government is bad because I don't like what the government is doing" doesn't really do it justice.
Or is there something specific to Google's Maven project that makes it shameful, while other types of military commissioned projects might not be?
I have a sneaking suspicion that Mr. Schmidt's views on what constitutes "killing people incorrectly" and what is required to "keep this country safe" both differ fairly significantly from those of the signatories of the letter.
Don't most of us want less war? Does it really drive you to enhance the capabilities of those committing acts of violence? How does one start to believe that by supporting the military you are somehow working towards a better world?
If you're not going to be an idealist, then in all likelihood you're helping to work towards the vision of an idealist with more power than you.
It's also possible that when a weapon is deemed technically capable, people will be more comfortable with broad use of weapons and with expanding lists of individuals targeted for extrajudicial killing.
You do not need advanced technology to build simple rockets with simple warheads and wreak havoc. These rockets are guided by nothing better than pointing and shooting. We see this in use in various paramilitaries around the world with little concern for civilian casualties: in Gaza, in Syria, in South Lebanon, in Yemen. If the West was similarly limited to such crude munitions then the civilian cost to destroying legitimate military targets would be enormous. Better guided munitions reduce civilian casualties and as such working on making munitions better-guided is indeed ethical.
If you think that not giving generals the tools to better avoid civilian casualties means they will simply not wage war in ways that endanger civilians, well history is pretty clear that this is quite erroneous. Giving generals the tools to wage war more ethically allows them to make decisions with fewer compromises e.g. civilian casualties.
This is not even to mention work on purely defensive munitions, e.g. Iron Dome, which is inherently ethical.
And if your opposition is rooted in a sense of pacifism, please, freedom is not free, it's not just a slogan. Anarchies (i.e. the lack of a sovereign government with a military defending territory) and their power vacuums result in warlords and the end of civil society. And that Malthusian outcome is clearly less ethical than almost any other. That's just human nature. Defense of civil society requires a military. Ideally, that military will act as ethically as possible while achieving its primary goal which is the stability (in face of external threats) necessary for civil society to flourish. There are two ways to make the military ethical: a) give it ethical tools so that it won't be forced to use less ethical means for achieving its raison d'être, b) set ethical policy. As sad that it may be that we have little control over the latter, as engineers we can have control over the former.
Our well meaning google friends might believe they are doing the right thing, but they are just doing what's fashionable instead of really thinking the consequences of these actions. Making ourselves weaker in the AI race augments the probability of war instead of diminishing it.
Our military doesn't get to pick the fights. They get to fight the fights.
https://web.eecs.umich.edu/~kuipers/opinions/no-military-fun...
From the introduction: """ Mostly it's a testimony that it's possible to have a successful career in computer science without taking military funding. My position has its roots in the Vietnam War, when I was a conscientious objector, did alternative service instead of submitting to the draft, and joined the Society of Friends (Quakers). During the 1980s and 90s, the position seemed to lose some of its urgency, so it became more of a testimony about career paths.
Since September 11, 2001, all the urgency is back. The defense of our country is at stake, so this testimony becomes critical. In short, I believe that non-violent methods of conflict resolution provide the only methods for protecting our country against the deadly threats we face in the long run. Military action, with its inevitable consequences to civilian populations, creates and fuels deadly threats, and therefore increases the danger that our country faces. """
Look up "Highlands Forum", DLI, MDDS. Sergey Brin was literally funded by NSA and CIA under MDDS.
http://nuclearfiles.org/menu/key-issues/nuclear-weapons/hist...
From the Soviet side, Andrei Sakarov.
For a country like USA it is stupid to waste human capital on fighting someone else's wars.
More and more funding in the military means more funding for specifically defense projects rather than straight up knowledge or public good.
What is the long term affect of this? Perhaps research far more focused on destruction instead of public good. More better drones, less general knowledge or cures for diseases?
It's a sad state of affairs, and workers standing up against military research within their companies is a good first step.
---
One of the first sources I found on this is below, but I've specifically heard about it being vexing from AI researchers, as many don't want to directly support military applications, but don't have much of an opportunities for funding otherwise.
https://www.bu.edu/research/articles/funding-for-scientific-...
https://www.nytimes.com/2017/08/30/us/politics/eric-schmidt-...
I wonder if Eric Schmidt left Google because of this.
Google should stop hiring activists and start hiring pragmatists.
When I was running a robot company, I made the decision to not do military projects (mostly for personal reasons) and it helped me hire some great people that I couldn't have otherwise.
Why go down this path at all? Google doesn't need to be a war contractor (I think "defense contractor" is a misnomer these days - we all know what this tech is used for: bloody violent and usually unfair/illegal war).
Google should avoid becoming a sponsor of death in the world. What happened to "changing the world" (for the better, I assume)? Is feeding the American war machine changing the world for the better? Really?
Google hasn't been doing that, ever. That's really the point of the story - the agency of the people who work at Google.
"Google Workers Astonished: Found Out They Work In Capitalist System, Not Nerd Commune"
This is too bad that the fear of retaliation exists for a letter like this. Is this actually the case? Or did the author read into them wanting to remain anonymous incorrectly? Perhaps they didn't want to become a spokesman for this issue outside the company, but are fine making their opinion known to coworkers and managers.
The fact that you are outing yourself (now and in your HN profile) is very unlike any other "spook" I know.
It's laughable that you say your " background is all out there for people to see".
Google/Alphabet is already a country in its own right, and libertarians would already be sceptical of Google's role in society.
This is just another step down a long road of Technocracy...
Also, wasn't "Don't be evil" dropped as a motto? I thought I read that.
Common misconception, and pointless to even bring it up. Because while "Don't be evil" is in their code of conduct, they regularly do evil.
The right course of action IMO would be to leave Google if you are unsatisfied with working for Pentagon.
But Google HAS to work for the Pentagon and especially on AI because the knowledge that comes out of this isn't only going to be military but to a much higher degree and relevant extent for civilized use.
Technology is always a double-edged sword and we have to learn to deal with the powers it gives us. But to not be on the forefront of technology when you can is a losers game and Google would lose much of its value if it isn't at the forefront.
On the other hand, this needs to be evaluated in the context of “balance of power”. Can Alibaba really refuse helping Chinese military. If not the Chinese are going to gain the power regardless of whether US has it or not. Any advanced country could gain such power.
These other entities may not follow the same ethical guidelines. Are we then proposing the US military should be less powerful than its rivals?
This does not ring true in my experience with "tech folks" in recent times. It may have been true at one time, but I think libertarianism in tech has fallen out of vogue.
Also, I take issue with distrusting how government (or any powerful organization) makes use of technology puts someone into the libertarian political philosophy according to this article. A healthy distrust of how powerful groups use technology makes a lot of sense for anyone and is definitely not something exclusive to libertarians.
If Google still continues to engage in the 'Business of war', would these 3,100 employees walk out?
Would they, then, also refuse to pay taxes to the US Fed Gov (which, obviously spends them, on the business of war) ?
Given that these strikes will occur one way or the other, isn't this stance actually adding more misery to the world?
(So do drone strikes in general. Much better to negotiate at a table with a mediator or use a sporting event to resolve disputes, but I don't currently run things here.)
Holy poopers, no. Drone strikes are not like premarital sex. The fact that we're _here_ (that being autonomous killer robots firing missiles from the sky to kill "bad guys") and no one bats an eyelash, means we're already in some kind of massive fail scenario.
We could, for example, go back to the rule of law. You know, before we believed that every bad guy out there was going to end civilization and deprive us of our "freedom." Before we decided blowing up wedding parties in Pakistan was such a great use of our time and money, and every Allahu-Ackbar-kaboom moment is a pervasive existential threat.
[1] https://www.nytimes.com/2018/04/03/business/apple-hires-goog...
A petition by 3000 employees (or, to be honest, 30) is enough to claim that it was in the company's interest to forgo this revenue to improve their standing among potential hires. Remember that tech companies live and die by the quality of their workforce.
Moreover, the idea that companies always have to act according to strict profit motivations is a myth. Just look at the billions corporations spend each year on charitable causes.
Yes, you could claim that these donations are done for PR purposes. But that would render your argument meaningless, because I can reframe any such altruistic actions in terms of PR. I have also personally witnessed many decisions that cost money but served some greater good and were never publicised.
Furthermore, if the workers really stood by their moral convictions, they'd use their collective power to tackle the issue more directly than by just appealing to Google. Companies like Google are like the railroads of the 19th century. They comprise the major infrastructure of modern society. A union at Google could threaten to effectively halt all institutional operations of governments or other companies in order to influence their actions. Is facebook manipulating voters? Okay, no more gmail for facebook till they stop. Is the government going to bomb Yemen? Alright, we're shutting down government accounts.
As others in the thread have also said, I'm doubtful many of these signatories will quit their cozy jobs if Google doesn't back down. Without the group solidarity and pressure from a union, most individual workers just don't have any good incentive to put their money where their mouth is.
But this seems an amoral thing to do, to me! A small group of unelected, unaccountable and non-diverse people abusing their relative power to try to force a democratically elected government or private company to do something? Why would anyone want that?
And just because employees at Google refuse, employees in one of the big Chinese firms won’t be able to. If our government does not have this tech then someone else will.
We live in this fantasy, like people aren’t dying everyday from war in so many places around the world. We are not immune to that.
Edit: to all the folks down voting me it’s a good idea to get a different perspective sometimes.
Wow, we really are insulated from our military today, aren’t we? Yes, the US government is still at war in Afghanistan and with ISIS.
I doubt this would happen, but I wish these only petitions would include counter-petitions, which people could sign if they don't agree with the petition, so we could get an estimate of whether this is just a very vocal minority of Googlers or viewpoints are split.
[1] https://www.statista.com/statistics/273744/number-of-full-ti...
It’s a sad state really but this is probably the main idea behind partnering with them in the first place
In this case what they seem to be doing is allowing the military to use some of the AI tools on Google cloud which are also available for general use. This is a much grayer area - it is one thing to not develop something specifically for the military, it is another to say that we will not provide the military something we provide to the general public.
Also, Google dropped the slogan "Don't be evil" in favor of "Do the right thing." Not that it matters because they barely followed it anyway.
"If we don't employ child labour, somebody else will! It should be us, so we can at least make sure our child-slaves get fed once a day!"
The fact of the matter is that Google has rare talent in AI. There isn't another company with their kind of expertise waiting in line behind them, and even if there were, sometimes you should keep your hands clean.
If you think about it, we (including Google employees) have known Google has been doing this in a different way to assist in waging war ever since Snowden showed us those slides. Google processes data for the government that allows them to know who to target for war.
Ease the lives of your fellow traveler.
Even Sun Tzu notes that warfare is not total.
It seems to me that 1st and 2nd amendment rights are both under particularly intense bombardment these days.
'Fundamental rights' (under US law) I think is defensible as fundamental is treated as an alias for constitutional so guns are in.
'Fundamental human rights' to me implies broad and fundamental applicability, i.e. the UN declarations and covenants on human rights, which don't iterate guns as a right.
Case in point, when non-US countries talk about human rights, none of them are talking about guns.
These drone strikes have been happening for decades without interpretive AI, and they will probably continue with or without. So let's make the strikes more precise and save more civilians.
>"But improved analysis of drone video could be used to pick out human targets for strikes, while also better identifying civilians to reduce the accidental killing of innocent people."
Suppose you decrease error rate by 10% (optimistic), but increase volume by 20% (probably underestimated). You'll get an 8% increase in innocent deaths.
Grow the fuck up - part of living in a democracy is tolerating the implementation of measures one disagrees with. Republicans might oppose birth control, but companies continue to support it in their health plans. Likewise, a state of the art offensive military is democratically wanted in the US, state your disagreement and tolerate it's implementation
And tolerating the opinions of others.