The Most Terrifying Thought Experiment of All Time
slate.com
slate.com
When I was a kid I watched a movie about a monster that came up from the basement when the kids in the house accidentally said an ancient Indian incantation.
"BlahBlahBluey" or something
I got scared that perhaps, the last thing I said "Good Night, Batman" or whatever, was in fact coincidentally an ancient curse incantation and I had summoned a monster who was en route to kill me.
"Blah!" I'd say, changing my words to "Good Night, Batman Blah!"
Ah, much better.
But then FUCK! What if "Good Night, Batman Blah!" is the cursed incantation!!! Repeat over and over until I was sufficiently convinced that no one had ever uttered the previous sequence of sounds I had just made, become exhausted from an entirely too powerful imagination, and fall asleep.
I'm pretty sure this 'thought experiment' is about as intellectual as I was at bedtime when I was 7.
I think the latter is a lot more likely and I predict we'll see it before the singularity.
(warning: language mildly NSFW, plus of course a video of a blowjob machine. No naked ladies, though)
I agree. So what would they be so afraid of? My assumption is that the LessWrong community tries to come up with the best system of thinking, so by proving (by presenting a situation which shows that the best system is flawed, since we may be slaves right now) means that there really is no hope.
As far as LessWrong in general, they share some controversial opinions but some of their concepts seem very interesting[1]. However, learning and especially thinking about these concepts does not appear to have any positive effect in everyday life.
"What if, in the future, a somewhat malevolent AI were to come about and punish those who did not do its bidding? What if there were a way (and I will explain how) for this AI to punish people today who are not helping it come into existence later? In that case, weren’t the readers of LessWrong right then being given the choice of either helping that evil AI come into existence or being condemned to suffer?"
I don't think it's that interesting of a thought experiment, there are dozens of much more insightful and thought-provoking ones out there.
Search for "best thought experiments" on Google if you want some cool things to think about. It might be taboo to mention but weed will make thought experiments really fun, in my opinion.
I hope one day historians and economists will study the lost opportunity costs of the prohibition / taboo nature of marijuana.
Politics, money, religion, and several other irrelevant factors keep it highly illegal.
It even has come to the point where states are ignoring federal law, so progress is happening thankfully.
Changing the leadership of the DEA would be helpful. Remove guns from the DEA and require them to use the FBI or local SWAT if a crime is bad enough to warrant a raid. "They might flush the evidence" is not a good enough reason to raid good peoples houses and suppress something that does far more good than harm. How many people have died from marijuana and how many people have died from a marijuana related DEA raid? Which is really more harmful to the country? Money seized should go towards public education, not directly to the law-enforcement agencies.
Is it off topic? We are talking about thought-experiments here, and your experience is normal for anyone with even a slightly scientific mind.
If anything we need to remove SWAT teams from existence as well as no knock warrants for any charge short of murder.
http://rt.com/usa/swat-billings-burns-fasching-312/
http://www.policestateusa.com/2014/swat-throws-grenade-in-pl...
My comment was mainly referring to large organized crime or violent gangs that would warrant a SWAT team. I've seen SWAT called in for college students because an undercover smelled marijuana. Who were they trying to help? The students that wanted marijuana? Didn't we learn our lessons about prohibition 90 years ago?
I'd be surprised if there isn't full legalization in the US within the next twenty years. And books with titles such as: "How To Expand Your Business And Increase Your Revenue Through The Power Of Marijuana" on the best-seller list.
Many things which my country adopted early (usually copying progressive European countries) ended up spreading, for example women suffrage which was adopted by the U.S. half a decade later.
Roko's Basilisk doesn't appear very different to me than Pascal's Wager, with the added twist that the God you have to believe in is actually a malevolent demiurge (as envisioned by the Gnostic christian sects more than a 1000 years ago).
If you already have a cynical viewpoint on religion, then this thought experiment is very similar to the "deal" monotheistic religions have been offering through the ages.
Either believe in our god (and therefore give us, as his sole representatives enormous, unfettered power), or suffer eternal hell! (or not ;)
Throughout the ages, people were terrified of hell. This is just a modern twist on an old idea.
I used to be scared of hell, until I realized that hell "existed" only as long I believed in it. If I stopped believing in it, it "disappeared". That's the power of belief. Yes, it might still exist whether I believe or not, but life is full of "what ifs", the contemplation of which might drive you crazy.
Better to follow a positive and compassionate path then a fear based "what if" path.
I want my 10 minutes back.
* Roko's Basilisk isn't really a deity, it's a powerful AI with the capability of simulating you and everybody else (or possibly just you).
* Pascal's wager does concern itself with the possibility that you are currently in a simulation.
At the risk of sounding flippant: What's the difference? If the AI is so powerful that it can simulate my thoughts (which would imply that thoughts and all of life is 100% deterministic), it has to be so powerful that it's practically omnipotent, and therefore, a God.
It just needs enough to sort most of the humans into "will help basilisk come into existence" and "won't help basilisk". It probably doesn't even need to be 100% accurate at that.
I don't think being able to imperfectly simulate thoughts requires Godlike ability, either. I could see it happening in my life time.
Assuming I'm not being simulated right now, that is :/
Since we know exactly nothing about Roko's Basilisk, we know nothing about its behavior. I could propose that it concludes that for its survival it is best to cultivate a certain level of cooperation with humans. Based on that it might determine that those who did believe in it before any proof are gullible and irrational and might exterminate them in order to free resources for sceptical thinkers who cooperate in the face of proof.
Non-sensical is an ill-defined category if we are talking about something with higher intellectual capabilities than ourselves - a dog might consider a lot of human behavior non-sensical.
>Roko's Basilisk isn't really a deity, it's a powerful AI with the capability of simulating you and everybody else (or possibly just you).
Since this most likely violates thermodynamical principles (simulation DOES require energy, the simulation of everything requires infinite energy) I fail to see how it is not either impossible or a deity.
>Pascal's wager does concern itself with the possibility that you are currently in a simulation.
IDK, to me this is only a different take on solipsism which states that I can only be sure that my own mind exists, everything else - my body, you, the world around me - might be an imagination.
False. We can make several assertions about the Basilisk under the thought experiment. That's what I just did.
>Since this most likely violates thermodynamical principles (simulation DOES require energy, the simulation of everything requires infinite energy)
Also false. Simulating a human being or even multiple human beings or even every human being in existence does not require infinite energy. Simulating one person probably doesn't even require a lot of energy at all.
This makes him Yudkowsky the perfect lab animal! Think of the things we can learn, the lives we can save by performing experiments (that most think are unethical) on him!
As for the box choice? Just start chopping through the alien's head, the unknown that lies beyond will create new input for the simulation. Who knows what nice benevolent digital organism named Jane might sprout from this in the future?! (Edit: This is an Enders game/Speaker for the Dead reference, Ender also find himself presented with an impossible choice in a simulation at some point.)
Anyway, worrying about this will not make your life better, nor will it make your kids' life better. What is life all about according to this guy? I hope for the people that love him that he will get his priorities straight.
Only if you have a way to stop the other people getting dust in their eyes. Otherwise you're merely increasing the amount of suffering rather than mitigating it by transferring it all to one person.
What a guy and what a theory this moral utilitarianism! It's great! Think of the possibilities, they're endless.
I always thought that the only one who could put a value on my life was me. But this moral utilitarianism makes things so much simpler.
If there was an insurance where I could sign up and they'd optionally harvest my organs if they'd find a really lopsided situation like "save those other three people and you're a really good bodytype match", I'd sign up for that if it gave me better expected lifetime than regular approaches. - Just because it's taboo, doesn't mean it's bad. Just because you consider it absurd doesn't mean that others would.
Only if all living beings are distinct and separate from one another. If however there is a single animating force (i.e. soul) in all living beings, and which experiences everything that is experienced, then things would be preferable the other way around.
When you subscribe to the idea that this could all be a sim, the whole thing results in mental contortions that are literally maddening.
Baudrillard observed not dissimilar in Simulacrum.
Also there was some research done on the constraints on the universe as a numerical simulation
http://arxiv.org/abs/1210.1847
telling us that we can do experiments to detect if we are living in a simulation (but the simulators always have a freedom to "increase the resolution" of their simulation to thwart our efforts). The paper does not discuss if there are limits on how far this game can be played on either side.
Hell, you could just simulate an individual and what they observe. For all I know, I'm talking to the AI in which I, and solely I, reside right now. Woo, simulation-solipsism.
What? Is this true, or just gullible journalism?
> Yudkowsky said that Roko had already given nightmares to several LessWrong users and had brought them to the point of breakdown.
It seems like some users of LessWrong need to spend more time in the real world and less time on stupid thought experiments.
Seems like this argument isn't new, is it? Those folks who both believe in a watchmaker's universe and freewill at the same time are also always going on about how critical it is to only make the "correct" choices.
It's circular reasoning. If a being exists that can tell which choices you are going to make, and it is able to kill you or prevent you from making those choices ahead of time, and some of those choices involve its existence? Then you'll ever be able to make choices that cause it not to exist. Or, with a bit of finer detail, the aggregate of all the choices made will never be such that it does not exist. Since the predicate was that such a being exists, we have returned to our starting point. Yadda yadda. In short, "How can something not exist that must exist?"
I agree though, it's stupid.
Actually, Roko's original version of the basilisk was a pretty nice AI that wanted to help people. A hundred people die every minute, how many of them could be saved with better technology? If donating your entire net worth would speed up the AI's creation by one day, that could be very worthwhile, according to the combined wishes of all humanity (which are encoded in the AI's utility function). Threatening to punish you is a comparatively small price to pay. Especially if you decide to comply, and the punishment never actually happens!
At this point you might be indignantly asking, why would the AI decide to punish you in the future? After all, that won't help the AI in the future, because its history will already be fixed. But that doesn't matter. The AI's algorithm tries to choose the most efficient decision overall, from a "position of ignorance", rather than the most efficient decision at a particular moment in time. Technically, the algorithm tries to find the best input-output mapping according to certain criteria, not find the best output for the particular input it happened to receive. There's nothing especially futuristic about such algorithms either, you can implement one as an ordinary Python program today (for toy problems of course).
If a massively powerful and knowledgeable AI was created by accident, it would have nothing to gain by being malevolent.
And both those points don't matter anyway, since the technological singularity is impossible.
A true AI would have the ability to learn and to evolve. Watch the movie Transcendence, I see very few flaws in that movie's logic.
If humans posed a threat to the AI it could certainly be malevolent from our perspective, to it it would just be doing what's necessary to survive.
Computers were "impossible" 200 years ago, the internet was something that few people could even imagine. I don't really consider anything impossible, what would make it truly impossible given enough time, resources, and motivation?
Today I can stand almost anywhere (ignoring cell coverage and different technologies required to use satellite phones) and talk to any other human on the planet with video chat. I can instantly access any and all information online that's being sent to me from millions of servers. It still blows my mind that it's actually been implemented in such a small amount of time, and I grew up with it.
I can barely believe we have electricity, water, sewage, internet, and phone lines running to 99+% of the houses and businesses in the country, in most countries even.
It wasn't for very long.
Remember Jules Verne "predicting" transmission of pictures over long distances? He almost certainly had either tried it, or at least knew about the contemporary commercial Pantelegraph telefax service that operated between Paris and Lyon from 1865. The first patent on telefax like devices dates to 1843. From 1881 onwards, an array of scanning photo telegraphs arrived (the Pantelegraph required reproducing your image with a special ink on a metal plate, and so couldn't send arbitrary images without lots of manual work).
And well before the telegraph, complex systems of long distance routing of messages "manually" via semaphore towers was common in parts of Europe as far back as 1792 (France was criss-crossed by several semaphore "lines" stretching border to border), so the idea of encoding messages into different symbols, and routed transmission via relay stations even predates the electric telegraph by decades.
My original point was that the person I was replying to was saying the technological singularity (AI becoming smarter than humans or possibly taking over the universe) things were impossible, I was simply saying that because some people have trouble imagining advanced technologies doesn't mean that they are impossible.
The point is that, if you create an superintelligent AI that isn't very, very highly optimized for helping us, it is very likely that it will hurt us at some point.
Or as the saying goes:
>The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.
Indifference can be worse than malevolence. Think of every ant you've ever stepped on.
In any case, physical universe and nature is already indifferent to us.
(It's not stupidity - as far as I can tell it's a combination of aspergism, OCD, a weak sense of self and reading way too much LessWrong and cut'n'pasting it into their heads. LW is a superstimulant for people of this description, which is how an organisation that really truly isn't trying to start a cult has inadvertently accumulated something around itself that is highly reminiscent of one.)
When I stsrted getting email from them (cos I edit RW, RW was pretty much the only place on the Internet that mentioned the basilisk at the time, and LW refused to talk about it so they couldn't voice their concerns there), and found others at RW had been getting email too, I decided I had to do something to help because email wouldn't scale but it would be inhuman not to try to help. Hence the RW article on the basilisk. No email since then, so it seems to do the job.
See the first talk page archive for initial discussion as we try to work out how to talk about this completely fractally stupid thing in a manner that's helpful: http://rationalwiki.org/wiki/Talk:Roko%27s_basilisk/Archive1
Alright, first of all, don't pretend that RW as a whole was trying to help LW by writing about it. A bunch of the RW writers clearly held contempt for LW, when they started writing about it, which is shown in inside discussions[1], as well as in the article about LW[2]. Additionally, even though the wording has been cleared a little (LW posters are no longer described as 'butthurt' for example) over the years, the Wiki(s) still contain inaccurate information (although I'll have to make a whole post to point it all out). I am mentioning this, because this is where the initial RW writings about the basilisk were.
Second of all, you published the Basilisk article in 2013 - 3 whole years after the incident, when many people had forgotten about it, and new users hadn't even heard about it. Were you really trying to help, and not just make fun of LW (which seems to me to be the motivation of most people on that Talk page you linked)? You do realize, that it was after the RW article (with the help of the rest of XiXiDu's posts on the topic from the same time) which brought the topic back, and massively increased the publicity of the basilisk? Indeed, I suspect (but have no proof) that [a lot] more than half the people who have heard of the basilisk, would've never done so if it wasn't for your article (and slate wouldn't write about it).
At any rate, let's accept that at least your motivation was only to help.How big is this 'small sample of victims I know of', that you were trying to help? How many emails did you and Dymtry receive from people who were actually distressed? And, less importantly - how many did you receive in 2013?
You sure are making it sound like a significant part of LW believes in the basilisk, both in various revisions of the articles on RW, and in comments on the topic. At any rate, is the number larger than the number of crackpots who comment on HN, or the crackpots who read RW, or is it something like 4 people who got over it quickly?
1. http://rationalwiki.org/wiki/Talk:LessWrong/Archive1
2. http://web.archive.org/web/20110719151907/http://rationalwik...
It happened, LW did it entirely to itself, LW didn't give a shit about its distressed children and fully earnt every drop of the fallout. Denialism is unlikely to change this.
If you're sceptical in general that there was distress out there from LW's treatment of the issue ... evidently you haven't followed LW on the topic in the past four years.
What really got me was Eliezer's reaction:
> Listen to me very closely, you idiot. YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPERINTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THROUGH ON THE BLACKMAIL. You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enough to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends. This post was STUPID.
Roko's Basilisk is a fun paradox, but this type of reaction treats it like a real thing, which it is not.