Hacker News Reader

In a lot of these scientific do or don’t debates - people tend to disqualify that there’s a consequence to not building the technology.

In the H-Bomb example, what would happen if Oppenheimer didn’t help develop the bomb? Russia most certainly still would have. And the world would be a different place today.

China, Russia, shit - North Korea are all working on super shady projects that you not I would want any part of. However that doesn’t stop the technology from being developed - and in some dire cases - not making the technology domestically puts us at a disadvantage.

This isn’t me saying that Amazon should continue providing Facial Recognition software to the Government. This is just saying never doing evil isn’t always the best solution - on a global scale.

Developing an offensive weapon is different than working on domestic surveillance technology.

Personally, I prefer a world where China has big-brother technology that the US does not.

Facial recognition isn't exclusively a domestic surveillance technology. For example, it could be deployed at national borders, with satellite imagery captured in combat zones, and so on.

Nuclear weapons have minimal potential for domestic use, short of a geographically partisan civil war. Facial recognition conversely has huge potential for domestic use, and arguably little potential for foreign use in a major conflict. If anything, I would say it's only potential for use in conflict would be to improve the ability to quash resistance in an occupation, which is more destabilising to global security.

Absolutely. An offensive weapon might be used to unfairly isolate and target innocent people.

Domestic surveillance will be used to unfairly isolate and target innocent people.

We all know this to be true. Therefore we should be much less apprehensive about developing offensive weapons.

This kind of argument can be used to justify working on all sorts of terrible things, but the premise just isn't true.

Most projects fail, and the people who work on super shady projects aren't some kind of evil supergeniuses. "Good" people are often better at working together. They tend to have more access to outside research and researchers.

It's also hard to know how far along these projects are. It's in these countries' best interests for you to think they're as far along as possible. The reality may be that you're building destructive technology to pre-empt something that doesn't even exist or will never work.

"If we don't do it, someone else will". There is probably no other argument more responsible for bringing us to the brink of nuclear holocaust than this one. This is the argument that convinced the otherwise pacifist Einstein to write to president Roosevelt, urging for the Manhattan project.

So what's evil about it? Nothing by itself, it is self-evident that we can not allow our enemies be more powerful than us. But there is a second step, which has to follow: those powers have to be used for a universal good which go beyond our selfish identity interests. If not then we become the enemy ourselves.

I believe the reason that many are protesting against corporations handing over their technology to the military is because they feel this second step is missing. Right or wrong, they feel that the state has lost its moral compass.

But wouldn't the correct response be to rectify the state/govt, rather than stop working on the tech?

Just because some select group decides that it's their moral imperative, doesn't mean everyone capable of the research has the same feeling, and therefore, any Govt who wants to stay ahead of the pack must be doing the research.

There's also the issue of it almost certainly being used against those governments that refuse to build it. So any decision not to proceed must consider that it's empowering everyone else.

> In the H-Bomb example, what would happen if Oppenheimer didn’t help develop the bomb? Russia most certainly still would have. And the world would be a different place today.

Russia (or well, actually the Soviet Union) stole the atomic bomb plans via its spy network:

"Although the Soviet scientific community discussed the possibility of an atomic bomb throughout the 1930s,[4][5] going as far as making a concrete proposal to develop such a weapon in 1940,[6][7][8] the full-scale program was initiated only in response to the intelligence reports collected by Soviet intelligence through their spy ring in the United States on the secretive Manhattan Project." [1]

That's the problem with your reasoning ("but they are doing it as well" or "they will do it as well"). Your reasoning is what escalated the Cold War and almost brought us WWIII.

[1] https://en.wikipedia.org/wiki/Soviet_atomic_bomb_project

It's a stretch to say the Soviets stole the atomic bomb plans, if what they got was a rough blueprint of one of the original designs which was not used to produce the size of bomb needed to be useful. The spys may have accelerated the development by a few years at most. Also I believe a lot of the intelligence came a lot from Russia's spy network in the UK, but semantics.

It's generally assumed, by Oppenheimer very vocally, that once an atomic bomb would eventually be produced, the physics weren't that complicated, this is coming from him, not me. This was accelerated by the U.S. showing that it could be done, but would not have prevented the development of a bomb. The U.S. couldn't keep researches from publishing fully, eventually the dots would be connected.

You state that this is what escalated the Cold war and almost brought us WWIII, well, the cold war was scary and developed lots of proxy wars, but it led to a 60 year time period where two of the largest powers never engaged in direct conflict. It's argued routinely that the only reason for this is both sides have nuclear weapons and the mutual assured destruction idea. If neither side had developed nuclear weapons then there would have certainly been another war between U.S. and Russia.

It almost escalated, multiple times, where it boiled down to someone in lower chain of command refusing to take the destructive action.

The book and series The Man In The High Castle (SPOILER) is essentially about the disturbed equilibrium between a nuclear and non nuclear power.

This misses the point. Which is that, 'everyone agrees not to do risky things' is not a stable equilibrium. Russia couldn't trust the US not to do it and vice versa.

That lack of risky things is a Fata Morgana. Risky things were done. The world was almost destroyed, multiple times.

> As an INDUSTRY, we have some very important decisions to make, one of which is, "will we work on this type of system knowing full well what it will be used for?"

Can't this easily be defeated by breaking the project up such that different teams are involved in different parts, and none of the parts know the whole thing? E.g. they could all believe they're all involved in some new super secret product development, but not know enough about the final customer or the other parts to know it's the US government.

Hence the "as an industry" part.

If you've ever read Snow Crash, the U.S. Government does something similar to exactly what you are talking about. That is why I've stated that at this point we can only delay the fruition of a state level infrastructure. Not stop it.

>The problem comes down to the engineers and researchers involved.

>As an INDUSTRY, we have some very important decisions to make, one of which is, "will we work on this type of system knowing full well what it will be used for?"

Its the same question engineers and researches who design bombers, and missiles, and weapons of all kinds should be asking themselves. Unfortunately there are always enough people devoid of morality and/or willing to defer to authority and facilitate the violence of authoritarian governments.

Morals have not worked work enough with advertising who's technology is only a hair's breadth away. The sea change you allude to will have to begin further down the food chain.

Too many people don't care. Just look how many people are plugging away at AI even though it could easily lead to extinction level events. Even before that people were making the H bomb and even other smaller scale weapons. Too many people are too greedy and don't care.

I just don't think this kind of opting out is going to do much. Maybe slow things down a little, but not that much.

But you're confusing the purpose of "caring". You are working under the assumption that the people who care would like to modify the situation in order to prevent these things from advancing further.

And of course, from that perspective you can make the argument that none of the proposed actions make any difference at all. Only outlawing the research itself has any chance of success for that, for obvious reasons. Now the simplest explanation is the explanation I'm afraid : if people are taking actions for reason X that don't have any effect on X at all, then perhaps there's simply another reason.

If you instead look at the reactions in this thread as "given this valid concern, how can I use it as an argument to exercise power over others ?", then you will find people's behavior surprisingly sensible.

People are discussing how to exert direct influence over how these big companies (where a lot of them work) move, and even more transparently about who should decide what. Who should decide who does things and what they're allowed to do.

Does it make more sense now ?

The same thing can be said about specific technologies.

1) Is it OK to work on facial recognition at all, knowing it will be abused.

2) How about machine learning in general. It is certainly being abused. Some people are worried about an AI destruction of humanity. Should all AI research be banned.

3) What about open source ML libraries? Having TensorFlow open source, certainly makes it much easier for bad actors to develop ML to do bad stuff.

4) What about open source in general. Having Linux as an open source, free software certainly enabled a lot of people to efficiently run computer systems that can be used for bad things. I am sure, that many of the surveillance systems used by the US, China, and Russia run some variant of Linux.

Basically, if you are writing software, especially open source, somewhere along the line, somebody will do something bad with it.