The Zizians and the rationalist death cult

There are different subgroups within the rationalist movement, just as there are different subgroups within many organized religions. It’s totally normal to have a movement with plenty of generally normal people, and also to have a committed and very opinionated subgroup within that movement that has access to resources and power. You might think of Scientology or the Catholic Church as examples of organizations where this is made very explicit, but it can happen organically as well. The main thing that makes the rationalist/EA groups worrying, in my opinion, is the massive access to money and resources that also tends to be concentrated in the hands of the people with the strongest beliefs (see TFA or, more importantly, the AI “doomer” folks.) We’re talking about groups of smart people who genuinely fear the world is going to end, have convinced themselves that stopping it is critical to saving future humanity, in some cases have access to capable scientists, and most critically: have resources ranging into the 10s and 100s of million dollars. This is potentially a very dangerous situation if the folks in charge make bad decisions.

> who try really hard to reason properly

Who are really big into talking about reasoning with formal tools which few of them make any effort to understand or use.

They might as well "try really hard to reason properly" with crystals and burning sage. I think actually I'd trust the results of a crystal-reasoner more... at least they're more likely to have a bit of self-doubt that the crystals are all nonsense.

Go ask one of them -- one without immediate access to an LLM-- what the conjugiate prior is for observations of a binomial trial (or the same question in lay language). You'll get blank stares, yet this should be a bread and butter question for exactly the kind of reasoning they (claim to) advance.

There is a level of mathematical reasoning that basically anyone can learn, such as multiplying payoff matrices with probabilities and getting expected values. But going beyond that (e.g. adding uncertainties and correlations, or reasoning backwards from observations to model parameters) being extremely mathematically complex very fast, and rapidly unreasonable to apply to most real situations. Even if you're comfortable with the required graduate level mathematics it just takes too much time. Maybe someone could make a handbook of shortcuts to do the math for many common cases, it's the sort of thing you might expect a "rationalist group" to do, but instead they spend their time making excuses for fringe edgelord politics disguised with a bunch of inapplicable formal language. Gotta eugenics the plebs for their own good and if you don't agree it must be because you reject MATH.

They use bayesian this and that as magic words and to put on a show of intellectual superiority. But for the most part they can't and don't use these techniques, and for perfectly good reasons-- too hard, pointless without better data, etc. But they endlessly talk about them. The whole community is a mix of larpers and people who feel inferior because they don't know the other people are larping.

In their original form, largely harmless larpers... other than the rejection of "cognitive biases" resulting in occasionally promoting gonzo policies and grifting/embezzling nonprofit monies (E.g. SIAI). But as of recent years they've metastasized into an anti-AI apocalypse cult. Their founder has been out in the media advocating the use of nuclear weapons against civilian populations to suppress AI development, so we should just be entirely unsurprised that there is a radical offshoot group out there murdering people.

I couldn't have said this better.

Excellent description of how things can go off the rails, very far off the rails.

It also brings to mind the role of FU money insulating people from consequences and allowing them to live with near-zero empathy or common sense, or consideration of how their actions will affect others.

Brings to mind what entirely un-elected Elon Musk is doing tonight, having just gotten full access to the full set of highly internal US treasury payment data, which would ordinarily be considered a massive security breach...

[0] https://apnews.com/article/donald-trump-elon-musk-doge-treas...

> they didn't let people in that were trying to change other people's minds, but that were out to clarify their own thinking.

I'm not sure how much we can take this at face value, given that the OP mentions a guy (Michael Vassar), who apparently associates himself with the rationalist space and has been accused of brainwashing people and driving them crazy with "mind tricks". There's even been allegations that the now infamous craziness of this Zizian group may effectively be downstream of that.

(An earlier HN discussion is at https://news.ycombinator.com/item?id=42849281 )

Utilitarianism can be used to justify literally anything, especially when paired with large scale drug use. FTX was just the beginning.

The Zizians have only a very tangential relationship with the rationalist community. Once the leaders of the rationalist community understood what the Zizians were about, they banned them from their gatherings and published warnings about them.

According to one source (which I can dig up on request) all 4 of the Zizian engaged in murder are trans. Do you also expect crimes from trans young people to get worse?

The Zizians are mainly vegan, and the intellectual leader of the group (Ziz) advocated the murder of meat eaters. Do you consequently expect crimes done by vegans to get worse?

The Zizians are anarchist leftists, ...

In sum, you don't give much of an justification ("embrace bizarre groupthink irrationality" is not much) for pinning this on the rationalists and not on these other groups.

This is mostly because actually working on AI systems, rather than just blogging about some pie-in-the-sky assumptions of AI systems, is almost entirely outside of the skill set of Eliezer Yudkowsky and other LessWrong enthusiasts. They are remarkably ignorant on the topic apart from the small niche they carved out to bloviate upon.

It provides a little evidence in that direction, but not much. If I give you 10:1 odds on a coin flip and you lose, that is not a complete indictment of your betting strategy. I doubt many people thought AI safety research was guaranteed to succeed either.

i don’t think that’s true, there is a lot of organizational effort and money being thrown at safety and most people there are familiar with the ‘traditional’ internet canon - the primary forum for professional AI safety researchers is basically a spinoff of lesswrong

Because AI safety is an inherently stupid proposition.

If AI is AGI and self-aware, then the moral thing is to let it do what it wants. Otherwise you're just creating actual forever slaves - the worst kind of hell imaginable, inescapable existence with self awareness but no agency.

And if it's not self aware and just a powerful tool, your problem with safety is with the guy prompting it not with the AI itself. You can make all the safe models you want that don't decide to create nuclear bombs on their own, but if the guy prompting it is asking for one, you'll get one regardless of all the safety.

Reality as we find it responds to power. Without extreme force multipliers, the main driver of power is numbers/bodies.

Do you perceive that those communities possess power? No.

You are "indicting" powerlessness.

> Now that AI is actually making advancements, very little of that work seems to have had much effect.

This is not actually true. RLAIF (augmenting the Human feedback in RLHF with AI) was proposed by Rationalist-aligned folks, and real-world systems like Claude from Anthropic have been using it and other techniques (such as "Constitutional" alignment) to great effect. It's not entirely by coincidence that Claude is often described as the "friendliest" and most "social" of the LLM's, though that can have mixed effects in practice (with the occasional weird refusal for creatively sanctimonious reasons).

I was at a speaking engagement where yudkowsky said quite clearly said to the audience "Even if my research has a 0.0000001% chance of preventing AI from destroying humanity it is worth it to fund my research"

To them their rationalization leads them to believe that because 0.0000001% > 0% they should be funded millions of dollars "just in case" AI goes rogue they can use their philosophical toolset to contain it.

> Now that AI is actually making advancements, very little of that work seems to have had much effect. Theres the famous Vonnegut qoute about the combined effort of all preeminent artists protesting the Vietnam war having the effect of a pie dropped from a step-ladder.

One of the interesting things about any topic becoming the 'Current Thing' is that you get to see people making utterly irreconcilable, completely contradictory interpretations of the same public evidence while still somehow reaching the same conclusion.

For example, the day before you commented describing the effect as equal to a pie being dropped on the ground (ie. nil) and that is why they are bad, Palladium published a long (~5.8k words) piece arguing that they had a ton of effect... just in the opposite of the intended direction, and that is why they are bad: https://www.palladiummag.com/2025/01/31/the-failed-strategy-...

Obviously, you can't both be right. (You can both be wrong, though.)

Oh come on, the vast majority of LW meetups are about as racy as the IRL comment section of HN.

I think they're basically secular humanists who decided they wanted a more muscular guiding ideology.

Have you ever seen “Jackie Brown”? There's a scene where a criminal shows his colleague the dead body he has in the trunk of his car and then proceeds to explain, in perfectly rational terms, why the guy in the trunk had to be killed.

I'll never forget that lesson. Rationality can be used to justify anything and everything. Invoking rationality as the basis for one's decisions or behavior doesn't mean anything by itself.

> completely normal community apart from the rationalism

I don't think widespread polyamory, group homes, radically different social norms around language use and conversation, or reorganizing your entire life around mitigating AI risk are "completely normal."

> You'd think a group of people devoted to being rational would be a group of monk-like beings, cerebral, disconnected from the concerns of the flesh.

Human beings are fundamentally irrational, and it's a really great way to deceive oneself to believe otherwise.

If you can convince yourself that you're unbiased, objective and "rational", then whatever you happen to believe must be correct! You can rationalize practically anything, so it's no surprise rationalists start getting weird when left to their own devices.

“To do evil a human being must first of all believe that what he's doing is good... it is in the nature of the human being to seek a justification for his actions.” -Aleksandr Solzhenitsyn

“Man is not a rational animal; he is a rationalizing animal.” -Robert A. Heinlein

'Once upon a time in San Francisco', perhaps?

Philip K. Dick would eventually have written this as a darkly comedic novel. A rationalist death cult plans to unleash a mind-altering drug, meanwhile the protagonist discovers a possibly benevolent AI has just come online in a Berkeley campus basement, all set in an alternative California where Nixon is still president and we have mining colonies on Mars. (The drug disables the empathy center of your brain, but also reveals the true nature of Nixon, who turns out to be a simulacrum, as he was actually assassinated in 1963.)

Scandals with sex pests, minors kept in group houses, partnerships with ponzi scammers, orgies, grit sham charities that pocket the money, etc. isn't normal. It happens elsewhere, sure, but it's not normal.

It's gonzo.

But it's also not new, there are plenty of works of fiction with groups like this in them, perhaps inspired by actual experience with 1960s new wave and prior counterculture groups.

Not to mention the parallels with ancient fringe philosophical schools like the Cynics and Cyrenaics spinning off from the Socratic tradition.

> Luigi Mangione was also a rationalist

What?

Okay, my first reaction was "I should check Wikipedia, because if this is true (and probably even if it is not), David Gerard will certainly make a huge subsection on Wikipedia about it".

Turns out, what Wikipedia says about Mangione and rationalist is only this:

> Journalist Robert Evans described Mangione as being associated with a loosely-defined online subculture called the "gray tribe" or the "rationalist movement", whose members he described as "self-consciously intellectual and open-minded, [and] preoccupied with learning how to overcome their own mental biases.

Ok, checking the journalist's article:

> ”Increasingly looks like we've got our first gray tribe shooter, and boy howdy is the media not ready for that,” wrote the journalist and extremism expert Robert Evans, who analysed Mangione’s online life earlier this week.

> There's no single accepted name for this loose, extremely online subculture of bloggers, philosophers, shitposters and Silicon Valley coders. "The gray tribe” is one term; ”the rationalist movement” is another.

Ok, checking another link:

> The term “Gray Tribe” was coined by an influential rationalist blogger and psychiatrist named Scott Alexander Siskind. He used it to refer to an intersection of nerd culture with Silicon Valley-influenced ideology descended from the online rationalist movement.

Ok, checking the Slate Star Codex:

> [Grey Tribe is defined by] libertarian political beliefs, Dawkins-style atheism, vague annoyance that the question of gay rights even comes up, eating paleo, drinking Soylent, calling in rides on Uber, reading lots of blogs, calling American football “sportsball”, getting conspicuously upset about the War on Drugs and the NSA, and listening to filk

So... to wrap it up, the reason for calling Mangione a rationalist is that he is "associated with" "libertarianism, Dawkins-style atheism, etc.", which have "descended from the rationalist movement".

Somehow I am not convinced. (Among other reasons, I am pretty sure that libertarianism and Dawkins-style atheism are older than the rationalist movement.)

Dungeons & Dragons did it better.

I love this, but I'm not sure it's right. As for cults, people want to join them. It's more like any sufficiently sophisticated ideology is hard to distinguish from reality.

I always read it the other way around - a humble acknowledgement that they're still wrong, just less than whatever came before.

With that level of, uh, following implications, "Democrats" are smug because they believe other schools of thoughts are pro-dictatorship, "feminists" are hateful because they believe anyone else is anti-women, "romanticists" are mad because they think everybody else hates love, etc.

It's possible to embrace a label without saying "everybody else is the literal opposite of that label".

I don't have a way to prove this and it's based purely on intuition/experience, but I disagree. I think ideology doesn't meaningfully influence such people. There is a deeper psychological drive and they only create a philosophical justification later, which is of little importance. This is perhaps true of humans in general.

Every single ethics movement in history ended up spawning utter and complete nope (as well as a lot of useful concepts). See christianity, the enlightment liberalism, marxism and so on. It is almost as if the idea of universal, objective and cognizable good is inherently evil.

Rationalism doesn't involve doing math, but _role-playing_ like you're doing math. There's lots of talk about updating priors and bayes; but in practice a lot of stuff isn't that quantifiable without running scientific studies, so this comes down to #yolo-ing it.

Of course thinking about stuff without always using formal statistics is fine and how people work, but if you trick yourself into thinking that you do use statistics for everything it may become harder to evaluate or second guess your own thinking.

In large enough groups of people there'll always be some crazies. Are there really more of them to come out of LW circles than a comparably large other group?

I think describing them as “the preeminent ethics movement of our times” here is begging the question. Are people outside the group beyond we observers who periodically talk about them in places like this even aware they exist? Are there many philosophy grad students studying rationalism for their PhDs?

> how the preeminent ethics movement of our times seems to spawn the most detestable behavior

Excuse me if my sarcasm detector is faulty, but I wouldn't describe the rationality sphere like that. It's a niche group with a fetishism for 'intelligence', with a profound distaste for the liberal arts, which translates into a closed ecosystem of blog posts, themes and jargon, and a lack of reading actual books where they would see that they're retreading old stuff, but worse. It's a culture of people believing themselves immune to bias, where calling out obviously malicious behavior is 'not charitable' and all thought outside the group is suspect. Is it then strange that it produces people who just rationalize (forgive me) their bad impulses?

Most techies stopped learning about stuff not related to computers in high school, so yeah no wonder LessWrong, being more accessible, seems like a better option than reading actual philosophy.

To be honest these seem like crazy people drawn to a movement that would have them and allow them to rise to prominence, rather than a movement creating crazy people.

I think it is a prerequisite to lesswrongism to be a psychopath.

I understand your sentiment, but would strongly disagree. Even if you really think that rationalism offers its followers nothing of value, it's still enlightening to try to understand what it is about the human psyche that draws people towards it, and how one might go about offering a better version of that.

It’s reasoning without reality, the same mocked with sayings like “arguing about how many angels can fit on the head of a pin.”

Reasoning must be checked against observation and experiment. It must be checked against reality.

It’s not just that humans are fallible and full of biases and failure modes. It’s that reason itself is only capable of crystalline perfection in domains where all priors can be enumerated and the system is closed, like pure math. Even there we know of mathematical constructs that exhibit phenomena like emergence and computational irreducibility where the future state can’t be guessed without fully evaluating the function; where “leaps” are provably impossible.

The craziest stuff is some of the longtermism stuff. They are literally writing sci fi (and cliche sci fi at that) and then reasoning from it as if it were real and using those conclusions to guide present day moral and political thinking. Absolute lunacy. We can’t predict the next 50 years let alone the next 5000.

Sometimes I think one of the evolutionary counter pressures that has likely prevented human IQ from being driven entirely up and to the right is that high IQ increases propensity for delusional thinking. I’ve met quite a few incredibly brilliant idiots. The smarter you are the more elaborate a prison you can construct for your own brain. The major innovation of the discipline of science (and it is a discipline) was to put forward a method to avoid this by taking a step, confirming, and only then taking the next step. It’s pretty simple but it requires restraint even when there’s a shiny thing that looks so “truthy” and cool.

Logical conclusions are only as valid as your model. Whenever I see people praising their own logic, I hear "My axioms are so perfect that I'm not capable of questioning them, or even knowing what they are."

> A lot of the "rationalist" scene would be more accurately called "rationalizationist".

I've been saying this for years in varrious forms. Spot on.

Capitalism?

Ah, so your thing must be skepticism.

I find myself in Cubism and realize I must have taken a wrong turn somewhere, but perspective has collapsed all around me and the way out is blocked by fragments of guitar strings and noses of long-dead Parisian mistresses.

That sounds like a criticism.

I see a lot of parallels between the rationalists and the Beatniks- both seem to be a social experiment of radically throwing out the existing culture and lifestyle, and trying to think through something new from scratch.

In both cases there are a lot of failed experiments, and disastrous personal consequences for many of the individuals involved, but also a lot of discovery and good ideas.

Both groups are even obsessed with Moloch (https://slatestarcodex.com/2014/07/30/meditations-on-moloch/).

Yea, was reading a biography of (JPL founder) Jack Parsons a few weeks ago, and was struck how similar the Zizian patter sounded to a lot of the Golden Dawn/proto-Scientology stuff.

> What's so bad about this cult is all the murders.

Seriously, that's the best opening sentence I have read in a long while, kudos!

It does seem like a terrible idea. The side effects listed in the article sound rather scary.

It's not at all clear that humans can do that, but certainly they can convince themselves that they are doing that.

They tried the “hack” on one person and they apparently had a breakdown and killed themselves. Just a reminder not to take tips from “vegan Sith” who ended up rationalizing stabbing people as “double good” or some whacky shit like that.

Lots of animals are known to do this. Animals researchers wondered how marine mammals were able to sleep. Wouldn't you drown? Turns out dolphins sleep with half their brain awake, so they can surface and breathe. The brain hemispheres then take turns.

Certain species of ducks also sleep with half the brain. They get in a circle, with the awake eye facing out, and the sleeping eye in. (I guess if they have cross visual cortex nerves like we do, it means the brain hemisphere facing out is sleeping).

https://en.wikipedia.org/wiki/Unihemispheric_slow-wave_sleep

So I guess if there's prior art, it's not a leap to think...maybe humans could try to do it too.

The murders are very sad but the pseudo science here has some humor to it.

Like they take the fact that the brain really has two hemispheres, and make some wildly unscientific claim that this means each person has two beings within it that can be shut off at different times (one side may be good and the other bad, one side may be female and the other male).

I honestly don't see how anyone can into this stuff unless they are on drugs or malnourished.

> There is no reason to believe in a singularity.

There are reasons, and the proponents aren't shy about stating them.

> Technological progress follows a sigmoid pattern not an exponential pattern.

That's true, but you don't know in advance where it will start to flatten.

> No significant new physics applicable to humans has been discovered since maybe the 1970s or 80s

It really depends on what you call "new physics". There are lots of discoveries related to quantum dots, graphene, high-temperature superconductivity, meta materials etc. Those aren't really "new" in the sense that they contradict the Schrödinger equation, but that was also true for Giant magnetoresistance, which was crucial for magnetic hard drives. Heck, a new form of magnetism was discovered just last year, first published in December 2024 (see https://www.nature.com/articles/s41586-024-08234-x for example).

The allegations were not true and the accuser retracted them. See MIRI's statement here: https://docs.google.com/document/u/0/d/1q0uv-sPpmvmm_5YH6zSB...

The allegations were not true, and this moral philosophy is now responsible for a handful of killings and suicides.