OpenAI Chief Scientist Has Been Making Strange Spiritual Claims | Hacker News Reader

OpenAI Chief Scientist Has Been Making Strange Spiritual Claims

futurism.com

20 pointsresource0x20 comments

The article on which this one is based ([1], [2]) sounds quite different.

What's reported to me seem more awkward motivational speech attempts, or jokes, than rituals.

Ths is the Sutskever part:

> Anticipating the arrival of this all-powerful technology, Sutskever began to behave like a spiritual leader, three employees who worked with him told us. His constant, enthusiastic refrain was “feel the AGI,” a reference to the idea that the company was on the cusp of its ultimate goal. At OpenAI’s 2022 holiday party, held at the California Academy of Sciences, Sutskever led employees in a chant: “Feel the AGI! Feel the AGI!” The phrase itself was popular enough that OpenAI employees created a special “Feel the AGI” reaction emoji in Slack.

The more confident Sutskever grew about the power of OpenAI’s technology, the more he also allied himself with the existential-risk faction within the company. For a leadership offsite this year, according to two people familiar with the event, Sutskever commissioned a wooden effigy from a local artist that was intended to represent an “unaligned” AI—that is, one that does not meet a human’s objectives. He set it on fire to symbolize OpenAI’s commitment to its founding principles.

---

[1] https://www.theatlantic.com/technology/archive/2023/11/sam-a...

[2] https://web.archive.org/web/20231122062339/https://www.theat...

The more I learn about OpenAI, the harder it is to avoid the conclusion that OpenAI is a straight-up cult.

It's a SFBA startup that is insanely mission focused, of course it's a cult. The difference between successful startups and cults is very small.

This is very different from the "cultish" nature that startups can have. I think this is an outright cult, with all that implies.

It seemed obvious to me for a long time, which is what was so scary about it.

OpenAI was always fairly overtly a cult, and an apocalyptic one, at that. (Or a manifestation of a broader cult, perhaps.)

I blame microdosing. Psychedelics are good, but you should do solid doses and then work on yourself internally. Socializing your mental symbols doesn't work because inner journeys are inherently solitary, so you just end up with cult-like behavior.

Also 'Feel the AGI'? Boooring. I would have been more impressed if they had gone in on 'Manifest the Basilisk'.

> Psychedelics are good

This is highly arguable. They're very popular with the tech crowd but I dispute that they're universally good. They seem to utterly destroy some people who come back from the trip as literal empty shells of themselves.

I didn't say they were perfect but I'm also not gonna write a medical disclaimer every time I express an opinion.

So occultism seems to be a thing with the openai leadership. I’m not sure if i wanna trust anyone anymore an AGI.

Occultism and cult-like behavior are different things. Occultism is a system for dealing with that which physics cannot explain. Cults can form around anything from supposed divine inspiration from golden plates nobody has seen to pony hypnosis.

Weird beliefs about AGI seem to be a common pitfall for people who spend a lot of time with chatbots of any sophistication. There were users who believed ELIZA was actually intelligent back in the 60s. It happened to Blake Lemoine more recently (speaking of what became Bard) and now it's apparently prevalent at OpenAI.

Arguably a factor of the personality type that wants to spend a lot of time with a chatbot.

I played with ChatGPT maybe thirty or so times, when I discovered that it would reliably return nonsense and incorrect answers to simple questions I moved on from it.

I think anybody that engages with it at length and tries to have 'serious' conversations with it has... well, I don't want to use the word 'issues' but... issues.

Someone plays a little too much 40k...

https://twitter.com/ilyasut/status/1578238338288402432

“This tweet was posted against the advise of wiser colleagues”

This reads like some self-aware humor to me.

Wild watching the op-ed against Ilya unfold.

there's nothing spiritual about this article or the claims. what a click bait title.

>"Feel the AGI! Feel the AGI!" employees reportedly chanted, per The Atlantic, a refrain that was led by Sutskever himself.

>The chief scientist even commissioned a wooden effigy to represent an "unaligned" AI that works against the interest of humanity, only to set it on fire.

I can kind of see what they mean, getting people invested in something through high energy group activities chanting slogans and burning effigies that represent the "bad, humanity killing AI" are tactics used to make sure everyone is thinking the same way and aligning with the "correct" goals. Feels similar to various cult-like crypto conferences and political rallies that mostly just stroke the egos of the people attending for being on the correct side/smart enough to invest.

I miss the days when they just chanted about Developers.

I can't wait to say I told you so :)

Reply on news.ycombinator.com