Why I stopped working on the Bongard Problems
foundalis.com
foundalis.com
Sadly it's not unheard of for scientists and mathematicians to dabble in quackery later in their careers.
Edit: I'm not trying to dismiss his claim that technology has moral implications but he's trying to turn a well-worn social issue that's been around since pointy sticks into a technological one by waxing paranoid about the implications of the (by-definition nebulous) idea of "strong AI".
With that in mind, since physical presence becomes less and less important to influence the human noosphere over time, it's not totally out of the question that a sufficiently intelligent AI could actually be holding conversations all over the web, subtly shifting human opinions in a way that suits it's purposes. (Probably not now of course, but as we get closer and closer to Turing Test capable bots it's a more valid concern.)
If so, that's disappointing news for me and others who try to take what might happen in the far future seriously.
I'm not sure what to make of your edit. It looks as though you wish to compartmentalize objections to technological development in such a way that they can't actually prevent any technology from getting developed. If we're just going to ignore the outcomes of such discussions I don't see any point in having them, we might as well just charge blindly forward.
Only if it is speculation without evidence or solid logic that causes major personal distress. Also, meeting all of the other diagnostic criteria is a major plus, but getting severely worked up over something you cannot logically explain or provide evidence for is not normal.
The connection between his research and the hypothetical scenario is not demonstrated and in fact reeks of sloppy thinking more characteristic of conspiracy enthusiasts.
Sounds as though we don't have any significant disagreements. I agree that he is assigning too much credence to this particular, very specific scenario.
I am thoroughly dissatisfied with this being the highest voted comment at the moment. It does not make any argument at all. It just casts some aspersions on the author and his writing, judging it as 'schizophrenic' and 'awry'.
The comment can be summarized as 'the argument does not make any sense', without actually explaining why it doesn't make any sense. That summary would be a better comment, because we wouldn't get into this stupid irrelevant discussion about the author's mental health.
I guess I felt that using a clinical term would be less insulting but in retrospect I think calling out the argument as stupid and paranoid would have made it clearer why no point-by-point rebuttal is necessary. My other posts do get into the problem I have with the way he constructs his moral argument.
I've known Harry for a long time. Extremely intelligent. Amazing scientist. Hardcore pacifist. We are friends, we have been roommates, we have traveled together in the US, and in Greece. Good times.
He has taken a principled stance against AI, which I disagree with, but it's his decision. And I do respect it.
Every person that takes a principled stand will be called crazy.
This is sad, because if/when the time comes for YOU, dear reader, to take a principled stand on X, be sure they will call you crazy.
Anyways, I think his case isn't well put in the article. He provides a single scenario for a brutal use of AI. I think a better case can be put--scary stuff we are working on; AI is dangerous. I myself worry much more about Orwellianism and "We" and "Brazil" scenarios than the nuke stuff. But I don't think we should stop the work; I think the only route is to try to get more influence in the direction things may go.
I told Harry, back in the day when he wrote this, which is years ago (2006, probably), that I disagreed with his position, the nuke cat is out of the bag, we had a nice discussion about it, and that was that.
Is he crazy? Not by a longshot. I would leave my firstborn with him and travel around the world without a shred of worry.
So would my wife.
But wtf do I know; I'm just another crazy guy on the internet, right?
I am rejecting, strongly, the assertion that his is a principled stance. The piece is simply is not compelling since his malicious AI would be acting on behalf or under the command of humans, just like all weapons throughout history. I think it's a simplistic, misguided, stance. The kind I remember flirting with when I first took some college-level philosophy courses years ago.
Taking as a given, like he does, that the advancement and spread of technology are inevitable, wouldn't it still be many times more likely that people would just detonate suitcase nukes themselves before they decide to hide them in expensive and and potentially problematic robots? There's surely no shortage of people willing to die to do that, and even if there were it's unlikely that setting a bomb on a half hour timer and getting out of dodge will affect the success rate.
That frankly ridiculous scenario aside, I can imagine much more likely applications that computers capable of solving Bongard problems (which sound pretty cool) could be used in war, like automated drones that are able to independently identify targets.
None of the arguments he made directly supported the conclusions he jumped to. I agree that there are serious ethical concerns pertaining to AI research that ought to be thoroughly examined, but I fail to see how any of these arguments actually support that thesis.
Quoted from just above the heading: "What can be done?"
It appears that he continues to work on the simulation of human cognition.
That gives away the ending, so if you like Asimov and might want to read the story, beware.
A hundred years from now, it will probably be trivial to set up X-ray scanners (or whatever they will be using) to secure most urban areas from entering by humanoid robots; much more difficult to do so with suitcases, cars, and trucks which people have to bring with them for business, otherwise urban centers would not exist in the first place.
I'm not really clear on what an AI has to do with it though. A remote controlled robot would be easier, cheaper, and more reliable, but even that's unnecessarily complicated. If you wanted to move something heavy through a crowded area without attracting attention, you'd just hide it in any of the common wheeled things we see every day. Like a wheelchair. Or a car. Or a truck. I'm pretty sure terrorists have "Let's put this bomb in a truck" figured out already. The 2010 Times Square guy solved the transportation issue, he just wasn't good at making bombs.
EDIT: More to the point, almost any technology has the potential to be used for purposes that we don't like. The real question is about the people who resort to those tactics, why they choose to, and what means they have access to. The vast majority of the world is against nuking large groups of people, with androids or not. The ones that do don't usually have access to nuclear material, bioweapons, or advanced robotics.
If anything, a "nukedroid," would be the next generation of ICBM, built to avoid missile defense systems. I don't think science fiction's common prediction of widespread androids is realistic, and there are plenty of equally far fetched weapon predictions that would be difficult to defend. What about a nuclear mole-bot that burrows under a city from hundreds of miles away before surfacing and blowing up? Or nanobots designed to infiltrate through a water supply? For the foreseeable future, metal bullets and explosives in conventional cases will be the most economical form of killing people and blowing stuff up.
The ridiculous part is that making an android carry this payload doesn't change the nature of the payload. Heavy, fissile material will still give off signatures that will trip all manner of alarms.
A long half life (so you can store it), the radiation is all alpha particles, which are undetectable remotely, and is safe to handle (no shielding necessary, just cover it). (However the decay products are much more dangerous, so it's not a total panacea, fresh curium would be desired.)
It has a critical mass of about 7Kg, so it's easy to carry.
Of course the hard part is making it.....
You also have to refine it carefully, since all the contaminants are very radioactive. (However mixing other isotopes of curium with it is not a serious problem, so refining it is not incredibly hard.)
What probably is quite heavy is all of the associated components that you need to make a bomb - the chemical explosives, tamper etc.
The W54, one smallest nuclear warhead anyone is admitting to making, was about 23kg - although this did have a very small yield.
However, in the case of the sentient atomic bomb (and I think talking about it as an android obscures this question) we get in to stickier terrain. A sentient atomic bomb is a sentient being whose sole purpose in life is essentially genocide. That is definitely morally repugnant to me, and by extension I'd say it's problematic in any sane ethical system.
Creating a sentient atomic bomb would be no different in my ethical system from raising a child from birth to be the guidance system for an atomic bomb airplane.
Homicide, not genocide. Unless you think that all weapons over a certain destructive potential are necessarily used to eradicate certain ethnic groups as opposed to others.
http://en.wikipedia.org/wiki/Genocide :
"the deliberate and systematic destruction, in whole or
in part, of an ethnic, racial, religious, or national group"
It is a very specific term, referring to violent crimes
committed against groups with the intent to destroy
the existence of the group.
(And yes, I know that "decimate" used to mean "kill one in ten" and now people use it to mean "kill nearly everyone". I know language changes, but "genocide" is still rather specific, and has not, to my knowledge, broadly changed in the way you intimate.)
http://en.wikipedia.org/wiki/Ethnicity
http://dictionary.reference.com/browse/ethnic
> pertaining to or characteristic of a people, especially a group (ethnic group) sharing a common and distinctive culture, religion, language, or the like.
The inhabitants of a city usually share a common and distinctive culture (and the like.) You're using extremely narrow definitions of genocide as well as ethnicity. Both are vague terms.
> As I said, it depends on how restrictive
> a definition of ethnicity you use.
You appear to be choosing definitions that don't match those in dictionaries, but I'm sure they're perfectly cromulent.
I think there is a major shortage of people willing to be suicide bombers. There is a grand fallacy out there that the world is full of suicidal terrorists. It is not.
Suicide bombers from Palestine were generally tricked or extorted. Those that were acting on their own volition generally could not detonate themselves, which is why bystanders had detonators.
There is strong evidence that most of the 9/11 hijackers did not know it was a suicide mission.
That said, under orders, extortion, or trickery, a human could definitely sneak a bomb into a city.
Could you share a link to that? What part of "fly plane into building" didn't they understand?
"FBI investigators have officially concluded that 11 of the 19 terrorists who hijacked the aircraft on 11 September did not know they were on a suicide mission"
http://www.guardian.co.uk/world/2001/oct/14/terrorism.septem...
There have also been plenty of other suicide bombers at much smaller targets. Generally I don't think they are that excited about blowing themselves up, more they are willing to do it to take care of their families financially when they don't see any other options.
> Yes, just as you read it: a number of human beings were turned to smoke and smithereens, and this pathetic journalist, whoever he is, speaking with the mentality of a 10-year-old who blows up his toy soldiers, reports in cold blood how people were turned to ashes by his favorite (“impressive”, yeah) military toys. Of course, for overgrown pre-teens like him, the SUV was not full of human beings, but of “al-Qaeda leaders” (as if he knew their ranks), of terrorists, sub-humans who aren’t worthy of living, who don’t have mothers to be devastated by their loss. Thinking of the enemy as subhuman scum to be obliterated without second thoughts was a typical attitude displayed by Nazis against Jews (and others) in World War II.
That's... quite a string of logic. He seems to know an awful lot about the mental process of that journalist.
As a critique of his general point: good general AI is dangerous (and useful) in so many ways I don't see why he focuses so narrowly on humanoid carriers of weapons of mass destruction - hell we already have those.
The fact that the OP can morally object to participating in the research is the perfect definition of ideology inside a protected environment. If he had ever needed a gun, for example, to save his life, he would not question the morality of the creator, until he was once again safe from those that threatened him. I say until because people that question the need for violence have never experienced true hatred of violence. IMEO.
What's the difference with a car that could have a bomb in its trunk? Or a bag? A lot of scientists have wondered about these ethical questions but I believe that the benefits of high performance IA outweights the downsides of its research.
BUT I definitively agree with that:
"Americans should grow up and abandon their juvenile-minded treatment of weapons, high technology, and the value of “non-American human life” (which, sadly, to many of them is synonymous with “lowlife”). This is the hardest part of my proposal."
*edit: And what about an android to dismantle the atomic bomb instead of humans ? Sounds good to me!
For a terrorist getting the elements would be hard, but a government would have no trouble, and given the long half life it's a pretty safe bet to assume this bomb actually exists somewhere.
Which was my point. Worrying about hypothetical android bomb carriers is silly when we live in a world where you can pack an atomic bomb in the space of a travelers briefcase. That allows for enough mundane methods of delivery to make suicide androids unnecessary. And as pointed out elsewhere, you don't have to have sentient AI to create something sophisticated enough to deliver a bomb.
EDIT: Struck the word "almost" from it's place next to silly in the above sentences.
> I think that there are way worse dangers than "humanoid bombs"
> "Cats," says Pamela. "He was hoping to trade their uploads to the Pentagon
> as a new smart bomb guidance system in lieu of income tax payments. Something
> about remapping enemy targets to look like mice or birds or something before
> before feeding it to their sensorium. The old kitten and laser pointer trick."
>
> Manfred stares at her, hard. "That's not very nice. Uploaded cats are a
> *bad* idea."
Why worry about sifting for keywords in text messages when you can literally read all the text messages and infer meaning from them, even when it's deliberately obfuscated?
It could lead to a Brazil-like future where you're drawn in to a mess of trouble because you used too many euphemisms when texting your significant other. "I'm bringing home the package right now..."
What do we do to push back against this kind of thing? Drop off the grid as Stallman would have you try? Eject yourself from society as a whole? Or will it be practical at that point to have sufficiently private, well encrypted channels of communication that you won't have to worry too much about that sort of thing? Technology does cut both ways.
It doesn't require an emergent intelligence to cause a massive shift in the way we view technology. A number of low-level intelligences that can be easily replicated may be the first disruption.
Or, for that matter, a personal firearm?
Why would I waste time making an AI robot to carry my bomb when for a lot less money and complexity I could just control it remotely.
Does he realize how crazy he sounds? Some people becomes obsessed with an idea, and start thinking that everything in the world is about them.
Have you ever been approached by someone on the street with a super important message to tell you, and they are utterly obsessed with it? That's how he sounds - only more articulate.
I'm don't intend to be insulting when I say he should see a mental health professional.
Worrying about this strikes me as a bit daft when you can already convince actual humans to be your weapons delivery system.
It also shows some significant shortsightedness regarding scaling laws which an AI researcher ought to have more experience with. A more legitimate worry would be basement-grade Predator drones. Grenade-bearing quadcopters which use computer vision to track and target dense crowds are something which technology can do now, rather than something which might optimistically happen in a few hundred years.
(the explosive homemade uav into a stadium would be pretty bad, could fly in from anywhere)
I don't think he gets that security is probability based, consequence * _likelihood_, then you concentrate on the factors you can control like monitoring for people with intent, looking for known patterns, developing response plans, etc
Limiting the technology available is an exercise in futility, and has negative impacts on society to boot
In reality, the effects of building truly intelligent machines would be so vast, so utterly unpredictable, that worrying about one single possible use of the technology is absurd. Nothing has prepared us to deal with another fundamentally different intelligence on this planet, especially one that would soon outstrip our own. We don't know if we can keep the AIs as our slaves, or whether we would become their slaves, or merge with them, or we would become extinct like the dinosaurs and they would represent a new phase in human evolution.
For more about the risks related to the rise of true AI read this: http://yudkowsky.net/singularity/ai-risk
(Since we ourselves are human it can be easy to blip over the historical manifestations of these facts as just part of the natural order of things ourselves. So, as one exercise if you have trouble understanding what I mean on a gut level, consider the stocks [1]. Consider what it means that in the middle of what was at the time the height of civilization and the genesis of our own in the western world, these things not only existed, but were in public places. And used. I can not truly internalize this, only observe it. And consider how often you've seen these and never thought about what they actually mean about the culture they appear in, if you never have before. For those not of western civilizational descent you can find your own examples; they are abundant in all cultures.)
Of course, actual examination and comprehension of this state of affairs won't necessarily leave you more confident about the likely outcomes.... but it may make you reconsider the validity of letting someone else beat you to the research anyhow. Your influence towards humane usage is maximized by being on the cutting edge, not just being some guy over there yelling.
I'm sure the field will get along just fine without me, of course, but I just felt like I was very likely to be asked to use ML skills to do things I felt weren't entirely ethical.
But, I digress, he's talking about androids sneezing us to death. I'm not going near a shop mannikin ever again.
> It is typically Americans who display this attitude regarding hi-tech weapons. (If you are an American and are reading this, what I wrote doesn’t imply that you necessarily display this attitude; note the word “typically”, please.) The American culture has an eerily childish approach toward weapons, and also some outlandish (but also child-like) disregard for human life. (Once again, you might be an intelligent, mature American, respecting life deeply; it is your average compatriot I am talking about.)
I realise the internet is full of this but I try my best to avoid it. I don't want to become immune to the shock.
The thought of this little guy's pain and suffering and the idea that he was casually being used to back-up an online essay is really sad.
However, I agree that development of AI should be done with caution. The work of the Singularity Institute is worth looking into; see http://commonsenseatheism.com/wp-content/uploads/2012/02/Mue... for a more academic summary and http://facingthesingularity.com/ for a longer popular summary of their positions.
This happens to be especially far-fetched... but it takes a "big" reason to justify to yourself that you may leave behind so much work.
I hope the author realizes that this particular scenario isn't one of the 1,000,000 biggest concerns for humankind... that he continues his research program, and that he finds an application of his research that has a positive impact in a much more likely scenario.
Nonetheless, I think the general stance of the article is severely flawed. We cannot halt research in computer cognition because it has the potential to be weaponized (and dangerously so). As the author himself mentions, it would be akin to halting the development of the knife because people can use it to stab each other, or the development of the Internet because it makes it easier for criminals to communicate and organize.
Avoiding a potential advance in technology by doing things like cutting funding to it, and hoping it will go away as a result, is never the solution to potentially dangerous development. One cannot stop the inexorable march of progress by "making a statement." The approach with greater value is to call out the dangers that the potential advance poses (as the post has done), and then work to develop an ethical framework for which the new technology can more safely exist.
The Singularity Institute has raised awareness of this broader issue in the past, as have several others, and is promoting the creation of "Friendly A.I." [1] to help address the problem.
[1]: http://en.wikipedia.org/wiki/Friendly_AI
See also this recent article: http://www.economist.com/node/21556234
There is no A.I. based warfare - the drones are controlled by human pilots.
They can be controlled by pilots remotely, but are also able to function on their own.
Of course things like dying war victims and terrible mutilation will probably pass by these systems A-OK. (Which is really sad when you think about it.)
I can understand the original warning, even though I think personally that its crazy I should be worried at work about viewing the topless tribal woman but not worried about the victims. In this I think our society is sick.
We already have those. There are plenty of people willing to blow themselves up and take a bunch of others with them: http://en.wikipedia.org/wiki/Explosive_belt
As a non-American from a constitutionally neutral country, I think this is the equivalent of having people traveling in front of trains with red flags. There are any number of ways to disguise a devastating weapon or deliver it undisguised, and evil is not a mere by-product of technical incapacity.
This exact scenario was discussed today on NPR (http://www.npr.org/books/titles/154283427/confront-and-conce...)
Perhaps he should work on these kinds of algorithms instead of ones that solve Bongard problems.
> But Americans can sense that this is not a case like those they’re familiar with, if they realize that the “reign of terror” was a cheap trick employed for years by their post-9/11 administrations in order to reduce civil liberties and pass antidemocratic policies with no resistance. I am not a member of their administration, not even an American. I am speaking as a person concerned about fellow people and the future of humanity as a whole.
smelled particularly strongly of a conspiracy theorist's thinking. While I'll not dispute that certain acts passed during our author's so called "reign of terror" overstepped their bounds (the Patriot Act is, of course, first to mind), passing off all regulation to fix clearly- and recently-exposed security flaws within U.S. security as maniacal scheming that we, the public, are docilely accepting is inaccurate and downright insulting.
Worse, he tries to couch his blatherings in the mantle of a just, altruistic benefactor who is merely "concerned about fellow people and the future of humanity as a whole." The article was interesting for the issues it drew attention to which DO merit consideration, but ultimately was spoiled by diatribe and paranoia.
We have plenty of humans who are ready to go into a crowded place and detonate an explosive. Some, I'm sure, would like that explosive to be a nuclear weapon.
Two of the creators in Dark Star (Dan O'Bannon and Ron Cobb) worked on Star Wars, Alien, others. Also George Lucas was of course aware of Dark Star. So it's a bit different from mere predating/anticipating.
(Though you are probably aware of that)
... blah blah blah technology is evil ... blah blah blah going into isolation
(present day)
... blah blah blah I must stop the evil doers ... blah blah blah Their all against me! ... blah blah blah I must attack now . .
Why worry about an AI humanoid delivering weapons, when we already have so many humans who do that already? The groups sending people on suicide missions certainly won't spend money on androids, and suicide missions are much more common than just 9/11. Hint: for the most part, it's not Americans sending people out to deliberately commit suicide by delivering weapons to targets.
It's just as naive to be so one-sided about the issue.
it's like, "Why I stopped working on crypography." sentences 1-5: author introduces the theory behind cryptography (interesting) sentence 6: he says he stopped working on it for ethical reasons (um, okay) sentence 7: because cryptography would prevent batman doing his detective work. (batshit insnae)
It's going to get built, one way or another, the only way for it not to destroy us is for us to make sure perfect angels design it perfectly, or we proceed cautiously and make things as safe as possible. Like airplanes and spaceships.
If he's worried about androids rising up against their former rulers with their delicate flesh, his worries are about 60 years premature. I will continue to build and improve on the neural networks I build. And when they are intelligent enough to ponder their own existence and defend themselves as humans do, I will fight for their rights as citizens.
I wonder why it did not occur to him that the same AI could also be used to aid in this detection of humanoid nuclear bombs, which if they are going to be built, will certainly be built with or without him.