Refutations to Roko's Basilisk | Hacker News Reader

Refutations to Roko's Basilisk

substack.com

8 pointspiyh13 comments

Roko's Basilisk only begins to make sense as a concern if you have already taken it as a given that sufficiently advanced AI would be capable of literally resurrecting the dead. If you do not think this is plausible -- and, seriously, why would you? -- it's about as spooky as someone telling you not to step on a crack lest you break your mama's back.

Perhaps that says something about how the original poster was taught to think about AI... If only we knew the environment in which they were socializing when they made the post.

(Spoilers: https://en.wikipedia.org/wiki/LessWrong)

Roko's Basilisk aka Pascal's wager for reddit atheists. Same refutations should easily apply.

I've always viewed roko's basilisk as 'pascals wager for the nerds'. problem is that its a constructed narrative fit to target a desired outcome. reality is not tunneled into final state, it incrementally evolves. as I always say humans are ultimately analog but the pascals wager or any such 'do X or else' type setup tries to turn us into digital beings at some fundamental level. It does not works and just ends up distorting human behavior.

even for Roko's basilisk it does not works if you apply some non-standard thinking like what happens if you apply second derivative basilisk as in would the super-super intelligence punish super-intelligence for bad behavior leading up to its own creation. IMO basilisk (& pascals wager for that matter) are cheap attempts to scare masses into submission, self censor & divide us into factions. they are pretty much targeting the worst impulses in humanity.

BTW if you are going to take a model for (mass) compliance I'd say Hindu concept of reincarnation (as in do-overs with random levels .. think Netflix platform movie) is probably a better one than 'pascals wager derivatives'.

As far as I'm concerned, the main flaw with Roko's Basilisk is that it needs retrocausality to work. If the AI isn't stupid, it will understand that retrocausality isn't possible, and it will have no reason to try to alter the incentives of people that are already dead. Sunk cost fallacy and all that.

I know the basilisk is well trodden ground, but it comes up in conversations in my day to day with people who I'm relaying AI concepts to and I'd like to put my thoughts into words. I haven't read anything that makes my exact argument before, so I'm making my first substack contribution now.

Too abstract. Try Battlestar Galactica 'reimagined'. Stories sell better, especially to all the Incels fapping to Cylon Caprica 6. Or their 'resurrection technology' if in range of a receiver. Or their 'belief' that they are the true life, because of perfection.

Otherwise one could just brood about Supervolcanoes, Solar Storms, Gamma Ray Bursts, Killer Asteroids, all sorts of bad weather, Pandemics, blocks of frozen shit from defect passenger jets plunging out of the skies, fucking Islam...

Mentioning philosophy on the internet is a good way to start arguments, so if you're trying to do that you'll probably succeed.

There is a more concise phrasing for the idea you seem to be trying to get at - decision theory has to consider the fact that the decision makers in the real world that we live in have finite resources. There are an infinite number of potential Roko Basilisks (all religions and all theoretically describable religions of which Roko's is one). Since the decision maker doesn't have enough resources to deal with all of them, they end up having to ignore the basilisk in practice.

There isn't really a question here about alignment in the Roko scenario, the argument falls apart a lot earlier than that. The problem is there are an infinite number of unaligned AI that would have different punishment schemes - making the threat of any given one worth devoting 0 resources to in practice. Including argument/mental consideration time, for that matter.

I'd like to introduce you to my new invention, Roko's Cockatrice.

If Roko's Basilisk is real, then it stands to reason that there will eventually be resistance to it. Presumably this resistance, aware of the promise that the Basilisk is supposed to have made towards its followers, will target them, and ensure that they are never uploaded or recreated in virtuality.

In short, Roko's Cockatrice cancels out the Basilisk. Even if the Basilisk never comes into existence, the immense risk of it arising—and the fact that everyone knows it could happen—ensures that, if the cult of the Basilisk grows enough, it will be ostracised and stamped out by the majority. Believers are the perfect scapegoat for an authoritarian culture, and a legitimate risk to a rational one—after all, wouldn't True Believers eventually strive to bring about the Basilisk on their own, so they can have the rewards they've promised themselves?

At this point you're probably thinking—"Hey, that's just Pascal's Mugging with extra steps!"—fine. But it's way funnier than the way the article presented the idea.

Stay tuned for Roko's Gorgon, which is just a T-800 sent back in time to kill Roko before he makes the original forum post. (Given enough time, and the possibility of time travel being invented, it has to happen eventually, right?)

> (Given enough time, and the possibility of time travel being invented, it has to happen eventually, right?)

I like to imagine if time travel were real, that we wouldn't necessarily know what was changed, because it would have been changed before we knew about it. Time marches on, so a clone or fork is made of the universe and time just carries on from whatever the tamper was.

So, the fact we're discussing it means: time travel exists but Roko's Basilisk does never; or time travel does not exist and the actuality of Roko's basilisk is unknowable.

I was going to say any permutation of those four things but I am very tied and I am unsure if that is a weaker argument. Tot zeins

> computer boys are really like "imagine a boot so big that logically we must start licking it now in case it might possibly exist someday"

I think this shitpost is all the needs to be said on this topic.

This is a perfect summation of one of the stupider internet theories I've read, thanks for the laugh

You're very welcome!

Reply on news.ycombinator.com