For tech CEOs, the dystopia is the point
bloodinthemachine.com
bloodinthemachine.com
Mark Zuckerberg has worked his ass off for like 20 years to go live in an underground bunker. Granted, it's a large and luxurious underground bunker, but this does not seem like an improvement.
Mark Zuckerberg controls the largest mass-persuasion device on earth - literally controls, he's got more than 50% of the shareholder votes - and is using the proceeds from owning the globe-spanning apocalypse machine that he's turned Facebook into to build himself a bunker underground to survive the apocalypse his machine is creating.
Just turn off the goddamn machine and you can actually enjoy Hawaii, Mark. All the good stuff is above ground!
You’re not going to convince people like Mark, but government can grind slowly, forever. No one else is going to protect you from these people at scale.
IF Zuckerberg turns off facebook, will Armageddon be averted? will it be hastened?
As long as 1 human in 8 billion is willing to make Facebook 2.0, apocalypse is still on. The only durable solution is for the majority of humans to moderate and control themselves. IF this seems farfetched, consider that self control and moderation has always been a prerequisite for human life and society, albeit with different subjects.
I don't know what a version of Facebook looks like that doesn't promote the apocalypse, but I'm also not the guy with his hands on the wheel, and that guy also seems scared of what it's doing. I'm also not particularly sympathetic to the "_someone_ would've sold those kids drugs!" argument.
FB deserves derision, but it is far from the sole reason that civilization is declining.
Cybertrucks arent what make the world of bladerunner a dystopia. AI isnt what makes the world of "Her" a dystopia.
Techies see dystopian Sci-fi not as cautionary tale which suggests technology should be avoided, but rather a cautionary tale about how humans should interact with technology, and human behavior to be avoided.
I think this can be argued to be an optimistic view of human capacity, choice, individual agency. Naturally, it conflicts with pessimistic views where humans have low-agency and are at the mercy of their environment.
One side thinks that when presented unlimited ice cream, people can learn to control themselves. The other side thinks that when presented unlimited ice cream, people will eat themselves to death.
That among humans rich with agency, there's inevitably someone who can see the opportunity in restricting access to the unlimited supply of ice cream as a way to control others and reduce their agency.
There are billions of people with "agency" navigating an unfathomably complex and dynamic environment that gains shape by the collective results of their individual actions. It's really not so binary as pessimism vs optimism, agency vs passivity, humanity vs environment.
Tech is irrelevant.
The problem is the combination of the people who want to abuse others, and the vast majority who allow it.
Or maybe better, the people who are willing to manipulate others, and the majority who are manipulable.
That problem exists regardless of tech and isn't created or even amplified by tech.
I say it's not even amplified by tech because techs power amplifying nature works for everyone. Every power tech gives to some asshole, it also gives to you and me to get around the asshole.
The only thing we can't get around is the complacency of everyone else who allows Amazon and WalMart to erase all the other shops. A douchebag figures out a button to press, like convenience and price, and most people happily just respond to that button without thought or resistance. And once enough of them do, the environment changes and no one else has any choice but to exist in the new worse environment.
The problem of most people not being skeptical and pessimistic and assuming the worst and rejecting shiny offers and propositions and ideas by default, resulting in them all being taken advantaged of and resulting in terrible systems that you and I have to live in, has always been, since there was no tech beyond mere language.
I think that is a real phenomenon, but different. The article seemed to focus on new technologies that are being brought into existence. It is one thing invent ice cream. It is another to figure out a mechanism to monopolize ice cream access.
If someone invents a new flavor of ice cream and monopolizes it, are they reducing the options available to individuals? What about if it is so good that nobody wants other flavors? What if the other flavors stop being made?
[0] https://www.penguinrandomhouse.com/books/592344/determined-b...
Neuroscience does not mean that everyone does the most degenerate thing possible at every opportunity. There are people that don't eat themselves to death when presented the option.
There is also neuroscience behind discipline, self sacrifice, and every other positive virtue you can imagine.
Bladerunner, Snowcrash, The Matrix, feature ‘cool’ characters and settings. It’s cool to imagine being a crew member on the Nebuchadnezzar or whatever.
No one references 1984, Brazil, or Children of Men to advertise their product, because those artworks create an uninviting, flat bleakness.
Maybe some blame should go toward authors who work hard to draw you in with style and vibe, in order to tell you a cautionary tale. Many in the audience probably miss the tale entirely, and are just immersed in the looks and vibe.
"English is a whore, sleeping with every language she encounters, and with a very successful sideline in pickpocketing, rifling through all their pockets for any word she can find", or however the old quote goes.
I actually just read the Wikipedia article about the writing of the script and Spike Jonze has explicitly said he was inspired by the false allure of chatting with a chatbot. So clearly the intent of the work is in strict tension with the GPT-4o demo.
I'm honestly pretty puzzled by the tech CEOs constantly referencing fictional dystopias. I honestly cannot believe they're ignorant of the artistic intent of most of these works. I reread Snow Crash after the original Metaverse announcement and I kept thinking "Did he actually read this book?"
I watched it, and it kind of was a dystopia. Not in the classic jackboots and oppression sense, but one of pervasive social disconnection and isolation. I mean, the main character's job was literally to write "heartfelt" letters to other people's loved ones on their behalf, because they couldn't be bothered to do it themselves.
Conversely, most "utopias" sound like they would be pretty horrible to actually live in.
One of the points of the Torment Nexus joke is that the Tech Company is totally oblivious to the fact that "Don't create the Torment Nexus" was describing something bad.
> Why shouldn't we say with a couple minor tweaks and a sexier name this "torment nexus" would be pretty cool?
I actually think your defense there actually gets to the core error: things can be simultaneously cool and awful. For instance: nuclear war. On the one hand, you have high tech rockets buzzing around insanely fast while precisely dropping incredibly powerful fireworks; which is admittedly pretty "cool," on the other hand, you have mass death and suffering. The Tech Company people are shallowly responding to just the "cool" part, and missing all the rest.
are there counterexamples? i thought the only difference between utopia and dystopia would be whether the speaker is promoting it or not
Also, to be fair:
- Siri had come out two years before "Her" was released to theaters, so the critique was more about voice assistants (which is a form of AI) than AGI or the AI sota.
- The closest thing to "Her" would be a Humane Pin with 4o or higher. A Humane Pin like experience that didn't suck would honestly be super cool. That would allow me to leave the phone at home!
Soma? Soda? Sofa...?
Not misanthropic. Masochistic. I will to be writing something more substantial on this soon, but the gist is; a lot of these projects are knowingly self-destructive, and what their leaders crave is for others to join them in their expressins of self-harm.
I disagree: The real goal is to get others to endorse and support their worldview about how the world works and what their own place within it is.
That doesn't come from the organizer enjoying pain/harm, but from desperately seeking to avoid experiencing a certain kind of it, emotional pain or harm to their ego.
That said, if someone else is willing to harm themselves (physically, financially, legal jeopardy) to "prove" their allegiance to the narrative, that may be considered a more powerful and pleasurable endorsement. Also a sunk-cost also makes it harder for the person to change their mind later.
In other words, more like "messianic cult-leader" and less "Pinhead from Hellraiser."
> That doesn't come from the organizer enjoying pain/harm, but from desperately seeking to avoid experiencing a certain kind of it, emotional pain or harm to their ego.
I think you actually describe a good picture of sado-masochism, the protective mechanism against emotions of unworthiness, though maybe you assumed I was emphasising the sadism side.
Of course no actual tech leaders are exactly like that (they're real people with their own subtle issues that are personal, private and none of our business - even if they would not allow the same dignity for everyone else). We are talking about archetypes here. Jobs was paradigmatic. And not limited to tech, things like drugs (and the media/arts) are a space where you can achieve mass-reach much more easily. For me, the portrayal of Richard Sackler (Michael Stuhlbar) in Dopesick [0] is an even better depiction of the tech leader archetype than the fictional Peter Isherwood (Mark Rylance) in Don't Look Up [1].
We talk about "disruption" as if it were a synonym for progress but what I see is how many tech-obsessed "creator types" actually delight in destruction, and the pain experienced by others who they affect.
Of course they would say, with that twisted Nietzschian faux-Christian 'compassion', that they "love" humanity and don't want to see anyone "left behind" (whether they want to be left out of an insane project or not).
The recent Apple advert was a timely expression of this. I'm glad they made it. Its better to have this "out" than ignored.
Another media tie-in here might be Atlas Shrugged. It too is a narrative to elevate an inventor--or more often investor--into an ubermensch hero role, one capable of projecting power against a dilemma that is mostly comprised of other people who failed to give the protagonist the respect/worship they were due.
I don't think it's a cynical strategy to sell products per se, but it comes about because the person with influence over the marketing is able to pick and promote a flattering story or framing.
____________
Also, as referenced in the article, the classic:
> Sci-Fi Author: "In my book I invented the Torment Nexus as a cautionary tale."
> Tech Company: "At long last, we have created the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus."
I've never read atlas shrugged and don't intend on it. But responding to this specific comment, if this planet had 0 creators and only bureaucrats, critics and consumers it would quickly go to hell.
Meanwhile, if we had no bureaucrats, critics, and consumers and only people that created things I'm fairly certain the world would be fine.
> Sci-Fi Author: "In my book I invented the Torment Nexus as a cautionary tale."
Just to be clear, we aren't talking about the torment nexus. We are talking about robots that can be interacted with using human language.