Dune's Butlerian Jihad and the Future of AI
technology.inquirer.net
technology.inquirer.net
but you cant ban the math!
the same transformer/attention ideas work great as compressors, function approximators, and surrogate models in physics, chemistry, CFD, etc., where they show up as "PDE‑Transformer", "Neural Operator", or "Hybrid Surrogate Model", and not as "chatbot". ;-)
So even if you outlaw certain AI uses, the core tech will just move into scientific and engineering workflows under different names, where most people won’t recognize it. Would be interesting whether it is possible to write a LLM-like program just using compression and function interpolation algoritms.
just my 2 ct
gzip can be used as a (not very good) LLM-like text and image generator: https://arxiv.org/abs/2309.10668
That being said I can very much recommend the two Hyperion books for a good look at AI and co-dependence in sci-fi.
I don't think we're talking about bans. The word Jihad, in Islam means struggle and usually it's accompanied by a religious "war", but not necessarily violent (see below).
> jihad , In Islam, the central doctrine that calls on believers to combat the enemies of their religion. According to the Qurʾān and the Ḥadīth, jihad is a duty that may be fulfilled in four ways: by the heart, the tongue, the hand, or the sword. The first way (known in Sufism as the “greater jihad”) involves struggling against evil desires. The ways of the tongue and hand call for verbal defense and right actions. The jihad of the sword involves waging war against enemies of Islam. [0]
Maybe people can't win against machines, but they sure can fight e.g. by refusing to interact with AI online, at their job, vandalizing mass surveilance cameras, etc. etc.
The last time there was serious public demonstrations against a technology was the anti-nuclear demonstrations in the 60s/70s when people were rightfully afraid that we had created something capable of destroying us all. The movements petered out, but it did force things like SALT, more aggressive IAEA to prevent proliferation, etc.
Right now the message is more or less: If you want to "AI-proof" yourself, pick up a trade or manual job.
If we create a society where all the peons will work in healthcare / trades / manual labor, while the trillionaires control all their other aspects of (digital) life, that could very well usher in reactionary leaders.
Much like the traditional Samurai of Japan got crushed [0] by a new model peasant army of the Meiji Emperor, which adapted European guns, artillery and drill and basically exterminated them.
no, the Bene Gesserit doesn't have prescience, the opposite maybe (postcognition)
>Spice melange, which can only be found on the planet Arrakis (aka Dune), is the drug that enables this enhanced mental cognition
no, for anyone not called Atreides spice just prolongs life; some mentats take sapho juice, but they don't need drugs. .
Via the Navigators, who lived in dense clouds of spice, transport across the Dune universe was possible - spice was as fundamental to trade and commerce as oil has been here on earth for the past century.
If that recollection was correct, it seems odd to overlook one of the key layers of the original work (as I recall it, of course).
Suk doctors are theoretically safer because of their imperial conditioning, but even that can be tampered with, as we see with Dr Yueh. This is a central problem in Dune: Whatever you depend on will gain power over you (or whoever controls the thing you depend on). Dependency on spice, on truthsayers, on mentats, or on thinking machines - even specific relationships. All of these are systemic vulnerabilities and therefore potential attack vectors in Dune.
(Edit: Suk Doctors get imperial conditioning, not mentats)
Dr Yueh was not a mentat, but a suk doctor who was subject to conditioning. (Which was broken by the Piter de Vries, via the pain amplifiers applied to his wife, Wanna.)
Paul himself was being trained to be a mentat, and there were no hints of conditioning there, neither with Paul, Hawat, or de Vries (albeit he was described as a "twisted mentat", whatever that means).
That doesn't necessarily mean they were grown in tanks, though, they could just have been "groomed for purpose" in the same way the Tleilaxu's matriarchal counterpart the Bene Gesserit did.
(The Tleilaxu are to me a very obvious stand-in for "patriarchy": there are a few men ruling over a genderless, fluid workforce, and women are literally just birth machines.)
Is it also true that Fremen are canonically descendants of Zensunni practitioners, which is a mix of Zen and Sunni Islam
Many other restrict machines, and the dogma is encoded on a religious text called the Orange Catholic Bible. And, this might be a big jump but, some of the specific anti-machine quotes are similar to Matthew 12:31-32 IMO
Dune is very much about systems of power and oppression, the obvious and less obvious forms they take, and the way they transcend and direct individuals. Even when the individuals are almost omnipotent - in theory.
The whole point of Dune is that these systems are human, but have an independent life of their own.
The narrative is about modes of human supremacy, and a jihad against AI isn't just some optional world building, it fits the narrative perfectly.
https://openai.com/index/model-disproves-discrete-geometry-c...
Likewise with math, it turns out, it appears. Likewise with most other things we classify as needing intelligence, I bet. It turns out that intelligence is not the same as humanity or sentience etc..
Go read some books from The Culture, better series than the Dune.
I think that most Amish were never used to technology though?
With a bit of click bait, this might be the first step of "Butlerian Jihad" -> "AI Crusade" and a founding text for the Orange CATHOLIC Bible.
I saw a take that you can now cite religious reasons to refuse working on and with AI - if more people try this, I wonder how it'd play out.
Interesting times.
Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc.
Additionally, the transhumanist ambitions represented by the Bene Gesserit or Navigator seem on par with all AI dangers to me. Why would the dangers of an incomprehensible machine mind be worse than the dangers of a transhuman being that was bred and optimized for generations? Who solved the alignment problem for these superhumans?
The gun, pointed to their head.
So I agree, I don't see why thinking machines are a worse alignment problem than enhanced humans. See also the Wrath of Khan.
I try not to think too deeply about it, or I won't enjoy the story. How could a society have gravity suspensors, "shields", and lasguns, but no advanced computing capability somewhere? And why use swords/knives for fighting, when projectile weapons are so much more deadly? I still love the books, but I have friends who can't enjoy the Dune world due to its many contradictions.
Edit: I realize that "shields" supposedly made ordinary projectile weapons ineffective - but they had advanced drone technology (e.g. the assassin drone used against Paul, which was also stupid)
The rest of the empire was too feudal and too religious to dabble into advanced technology. And the other players (the BG, the Tleilalxu, the Guild) were against it as well.
What good are drones, except as weapons of assassination? Can't use them in an conventional war, there are shields. And they're very expensive, remember, the Great Houses don't have advanced manufacturing capabilities (that's IX). And, finally, there are no great wars in the empire, most houses keep to their planetary fief; the Baron's attack on the Atreides' forces on Dune was an exception. It was also mindbogglingly expensive, 80 years of spice profit.
Presumably a drone could get close enough to push a blade through the shield (maybe use a rocket booster or some other advanced "suspensor" tech) at low-ish speed (a knife would have to travel through the shield on the order of a human reaction time, otherwise even knives wouldn't work. That's about 0.1s/0.1 m, so 1 m/s - ish.)
The propelled blade weapon doesn't need a control system in the "shape of a human mind" to be effective. A very simple control system would work. If they have ornithopters, then they have control systems. The blades could even work like hawk talons (so an ornithopter drone falling on a shielded fighter, then springing what is effectively a pointed metal trap.)
But more importantly, you are right that if the weapon moves too slowly, then you could just get out of the way. That's true of any weapon, no matter how it is propelled. That's why I said above you have to be able to penetrate a shield faster than a human reaction time, otherwise no weapon would be effective.
> Paul wasn't aligned with the Bene Gesserit, and hilarity ensued.
Hah, great perspective and I agree. Even normal humans are so difficult to align, no wonder it went wrong with a guy with prescient powers.
This implies their technology is unfathomably complex, too complex for the human mind to understand. Doesn't seem to be the case.
>Seems like if it was adopted as a policy, there would have to be many gray areas where people pretend that they aren't really using AI - the ship's computer is just a domain specific finetuned version, after all, etc.
Ships are using Navigators, who are trained in "higher mathematics" and are also prescient. They have no need for AIs, nor do they wish to court destruction if they are found out (I can only imagine the BG will have the Spacing Guild by the balls if they knew).
> Why would the dangers of an incomprehensible machine mind be worse than the dangers of a transhuman being that was bred and optimized for generations? Who solved the alignment problem for these superhumans?
The books address these questions, especially the one about a certain God-Emperor...
More concerned about America. Its freedom of speech means that every douchebag billionaire tech bro is allowed to run of their mouth and rile up the proles.
We aren't enslaved or at any risk just yet with this form of "thinking machine", but we do need to culutrally and emotionally come to terms with them.
Let's not have any other "jihad" just yet, there's plenty of other jihads going on right now.
> With the rise of AI came the rise of a technocratic class that created and controlled these machines, leading to oppressive structures over knowledge and the economy.
It also quotes Dune, where it is said more directly:
> Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.
Totally, but we can try to resist it. Technocrat is a perfectly fine word, and I'd hate to see it go away permanently from its current meaning.
It’s as if calling something AI is all that’s needed to justify it being the same thing as Skynet, Dune automation, Terminators, Neuromancer, and every other cultural trope using the label of AI.
The ultimate fact is that LLMs are a pretty powerful technology that has little to nothing to do with most fictional depictions of intelligent machines.
Sloppy thinking all the way down.
My thought about the turnaround: https://news.ycombinator.com/item?id=48343390
It annoys me to no end that people keep looking at the finger while pointing at the moon. No, we don't have Skynet today, but you can believe Sam Altman would have loved to see Skynet appear out of ChatGPT. That is the end goal, and it's worth discussing now, not when it's too late.
When talking about artificial superintelligences, you don't want to study their potential effect on humanity the day they appear, because it is too late.
That's not removing the amazing ability of LLMs and the scale of this accomplishment, don't get me wrong - it's incredibly impressive that specialised models are now able to walk down pathways for creative problem solving - but I don't think that suggests a sentience or profoundness - rather it's just number-crunching at a astronomically larger scale.
The people complaining about becoming stupider because of their AI usage might have something to say. Those that wish they didn't have to use AI but are coerced by their workplace might disagree. What of the day something negative happens to you based on the wrong, unreviewed decision without appeal by an AI agent? Do you truly believe this day will never come?
You don't need literal chains to be a slave.