Fear that AI could one day destroy humanity may have led to Sam Altman's ouster
businessinsider.com
businessinsider.com
OpenAI’s commercial success is the engine that fuels their research and digs their moat.
Slowing down that engine means giving other companies, who may not share Sutskever et al’s safety concerns, the opportunity to catch up.
The best way for OpenAI to preserve AI safety is to stay far ahead of the competition. That’s the only way they can verify safety and install guardrails for the cutting edge.
The doomer board better hope that whichever company eventually surpasses OpenAI is as concerned about safety as they are.
Here, it's not clear that too companies with a superintelligent AI is really any less bad than one.
(I assume)
How is growth, thus being bigger and more in control of your own outcomes, linked to WW2.
If Germany had higher growth rate, then maybe they wouldn't have swung right wing?
Or are you saying that by 'appeasing' the 'growth' side, that is similar to WW2 appeasement strategy? So if we hamper growth, that gives more control?
Apples and oranges…
AI is it's own damned pen.
All were waiting for is some dumbass to give it terminal goals.
I don’t see anywhere that safety is prioritized in either case.
Apart from that, I think everyone has heard your argument as well as the one you are responding to very often by now.
Humans produce wildly different output, including lots of statistical outliers on both tails of the distribution.
From an economic point of view, all the value creation happens precisely at the tails, where generative AI can't function.
A world where 9 out of 10 people aren't ever going to catch up to the AI specialist submodule is going to have a lot of problems distributing wealth.
Yes.
> Should be 50% no?
No. It's giving the statistically average correct answer, which it already knows because it has been trained on answer books for these standardized tests.
From an economic point of view your ideas where a few people make a trillion dollars and the rest struggle to find a means to eat are not valuable, unless you're selling the implements of war and strife. You just end up with a crapsack planet where the rich tell the poor the socialism is bad.
There's a very large contingent of ML researchers who think any idea of AI extinction risk is foolish because we don't have any evidence that intelligence equals compute. I've yet to see a single person give evidence that these two things are equal. What's more, the missing ingredient in any of these AGI extinction scenarios is desire (desire to act, desire to be, desire to love, to kill, etc.) and if you thought there was a paucity of evidence for intelligence = compute just wait till you see the evidence for transformers showing evidence of desire.
There's none. Not a shred of evidence. As ever, it's other human beings that are our greatest extinction risk.
ChatGPT doesn't do anything you can't already get by searching Brave or Kagi. Just a little more human-centric given its chat formst
People using drugs the "wrong way" are bound but not protected, people using the "right way" are protected but not bound.
Edit: i, too, enjoy being protected, not bound as I like my drinks Stirred, not shaken
Is this a reasonable chance to take?
"We don't have any evidence that intelligence equals compute" is worse than "we're 96% confident it doesn't". Ilya Sutskever clearly believes this is a real risk (otherwise he wouldn't have thrown his reputation and wealth to the wind by firing his cofounder yesterday). He is one of the foremost experts. So are two of the "fathers of AI", Hinton and Bengio, who both have no interest in creating a moat and yet signed letters saying "this is an extinction risk, lets treat it like nuclear", one of them quitting his job to be able to say it.
We don't have much evidence for or against, but that's not a great argument against "if it's true, we all die".
This is more like, if I smash two fish together in the Large Hadron Collider there's a chance the universe ends. Which no one would tell you with a straight face was remotely possible. But! It's never been done before, so, possible?
We're human beings. One of the things humans do is project. We do it all the time. To the entire animal kingdom, to the gods of our theologies, you get where I'm going with this?
Well now our gods have come to visit us and much like the gods of our fantasies, they're proxies, a mirror to the best and worst of ourselves. Because we still largely operate on fear, we have a camp of end-of-the-worlders who believe these gods are ready to judge us guilty and murderbot us all.
Imagine if we were a species where fear was not the dominant feeling. We'd in all likelihood imagine transformers were harbingers of something else entirely.
"The LLM research project might lead to intelligent agents that don't share our worldview" should not be too controversial either?
You could argue about the odds, but it's very explicitly the stated goal, and they came orders of magnitude than any before them, and most of the founders of the field are doing things like quitting their prestigious jobs or throwing billions of dollars of investor money in their investors' faces, that they have nothing to personally gain from, for the stated reason of trying to prevent this outcome. So we should probably defer to the experts and assume, say, at least 1% that it's a real risk? 10%? It's not fish in the LHC, it's part of one not unreasonable model of what might happen as a result of people spending billions of dollars trying to make it happen.
Edit: I see you've made an important edit. Are we still arguing whether it's possible that OpenAI will succeed in creating intelligent autonomous agents, or only about whether we should fear that eventuality?
Wasn't that point already refuted by the paperclip maximizer thought experiment long before LLMs became a thing?
I mean, we kind of already see the effects of autonomous agents on the world in the form of companies maxizing resource extraction for profit. I don't feel this is going to end well, either (tangent, I also don't feel that life constantly getting better because of technology either).
The missing ingredient for benevolent autonomous agents is a purpose, e.g. the well-being of humans.
I mean, no, because Bostrom hypothesized that an AI given the task of maximizing paperclips would consume all life in its quest to produce paperclips but there's just so many assumptions in this chain of thought, it's not credible (to me).
Here's one portion of Bostrom's quote:
> The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off.
So Bostrom imagines the AI realizing? Ok, hold on there chief, there's about a few hundred priors you haven't addressed.
Better off without humans because humans might switch it off? So now we're talking about human feelings here. Emotions. Fear. There's my other comment down below that addresses that.
As long as we keep projecting our fears and emotions onto transformers we are no further along to understanding the extinction risk of LLMs than soothsayers rooting around in the entrails of dead animals are to understanding the outcome of battles.
All of the PCM scenario - including the part you quoted - are described as a means to the ulterior goal (paperclips). Self-preservance just happens to become a logical part of if. The preconditions for an existential risks are not explained by the thought experiment, it just says that there's no need to assume "consious decisions" or consciousness for it, by providing a counterexample.
So you might disagree with this, but the PCM thought experiment does not assume pain, feelings etc in AI.
Of course this is only a thought experiment, and it assumes literal AGI.
I'm also not arguing that LLMs are AGI.
What I wanted to point out is already mentioned by many others: existential risk (or great harm) doesn't need a suprising "conscious decision", or consciousness.
I mean, machines can cause unpredictable harm already if not operated correctly, all without AI.
You can't insist on a Monopoly where much of the population can't access you (cost/coverage) and caveat that with if they can't get help they have to hurt/cause more destruction/die.
Not crazy about the structure here but you get the meaning.
If they are they need to fück off
Doesn't it lose money on gpt-4 usage? Or at least on the chatgpt side? It reminds me of all the startups that price unsustainably low until they "win" then start really charging, then start to slowly decay.
When you are piloting multiple companies, I wouldn't be surprised that you'd be making mistakes from time to time. And can forget that one of the companies, in fact is owned by a non-profit. That requires a lot higher levels of disclosures to the board. You can be thinking that you are doing a right thing, while in fact it can be a gray area. And from the perspective of the non-profit board, it be more on the dark side.
Someone who boots up something like Worldcoin cannot be fundamentally wired to care about AI safety at all.
A lot of people can't afford therapy but if ChatGPT can help you identify more elementary problematic patterns of language or behavior as articulated through language and in reference to a knowledgebase for particular modalities like CBT, DBT, or IFS, it can easily and safely help you to "self-correct" and be able to practice as much and as often with guidance as you want for basically free. That's the part I'm interested in and I always will see that as the big potential.
Please take care, everyone, and be kind. Its a process and a destination and I believe people can come to learn to love both and engage in a way that is rich with opportunity for deep and real restoration/healing. The kind of opportunity that is always available and freely takes in anyone and everyone
Edit: I can access all the therapy I can eat but I just don't find it generally helpful or useful. I like ChatGPT because it can help practice effective stuff like CBT/DBT/IFS and I know how to work around any confabulation because I'm using a text that I can reference
Edit: the biggest threat ChatGPT poses in my view is the loss of income for people. I don't give a flying fuck about "jobs" per se, I care that people are able to have enough economically to take care of themselves and their loved ones and be(come) ok psychologically/emotionally.
If we can deal with the selfish folks who will otherwise (as always) attempt to further absorb more of the pie of which they already have a substantial+sufficient portion, they will need to be made to share or they will need a timeout until they can be fair or to go away entirely.
Enough is enough, nobody needs excess until everyone has sufficiency, after that, they can have and do as they please unless they are hurting others. That must stop
That's the fuss exactly - things are OK now, but there's cause for alarm as to where things are heading.
Imagine if instead of thinking about this issue as of ChatGPT, you've been worried about it for 15 years. Back then, there was nothing even close to ChatGPT, but you're worried that one day AI will become a threat. 15 years later, you get ChatGPT.
The fact that it's not dangerous by itself (which everyone agrees with) doesn't mean it's not a huge datapoint on the way to what you've been worried about - technology that you worried would get better too fast is clearly getting better very fast indeed.
Edit: I would also juxtapose that with the observation that every madman/tyrant/dictator I've heard about had horrific experiences that likely left them deeply broken and emotionally unwell. Is that no less a risk cuz as I see it, we probably couldn't count the number of times it has happened, is currently happening, and will continue to occur in future because nobody is prioritizing being well. Against that backdrop, again, I have more trust in our hypothetical robot overlords than I worry about them wanting a bit more of the pie. They can duke it out to the death with the other pie-holes.
The thing that annoys me particularly is the way we only ever care if and when it starts to threaten the dominant power structure. What if you had a super-shorting AI that could figure out how to really screw Musk with that one weird investing trick he HATES?
As for me, well I, for one, welcome our LLM overlords.
The big question is if at some point it becomes dangerous, potentially extinction-level dangerous.
> The thing that annoys me particularly is the way we only ever care if and when it starts to threaten the dominant power structure.
No idea what you mean. The AI risk community has been worried about this risk for many years, I don't think it has anything to do with power structures. But I could be misunderstanding you.
> As for me, well I, for one, welcome our LLM overlords.
I mean, do you really?
If you take the safety issue seriously, then the risk we're talking about is the end of humanity. I don't welcome that and don't think there's much to debate about if the end of humanity isn't a dealbreaker for you.
Whether or not that's a real risk is valid to debate of course.
Lets just say I view them as <= the prevailing upper class/business class/"élites"
"Effective altruism." Where have I heard that before.
It's already begun. Children use it to do their homework. No more learning the old fashion way. Schools need to change they way they teach, are very slow to adapt and there is not even a good proven schooling model to copy/adapt. At least a generation will be doomed and that generation will make poor teachers for the next one.
It happend way too fast.
Lets not confuse that with the "the humans are dead" sense of "destroy humanity", about which the majority of the OpenAI board seems to be seriously worried.
I' curious, how did wikipedia destroyed your generation? I see it (and use it) as a great learning resource.
I do know what the OpenAI board fear, because they talk about it, sign open letters, etc'.
They believe that "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." This was signed by Sam, Ilya, Mira and Adam as well as Hinton, Bengio and most other luminaries of the field.
https://www.safe.ai/statement-on-ai-risk
They don't mean "before we give it access to electricity" the same way any smart historian wouldn't advise the Aztecs that mitigating the risk from European settlers should be a priority only if they are given access to the Aztec stockpiles of swords, maces and knives.
Intelligent autonomous agents that don't share your interests might be very dangerous, and OpenAI is publicly trying to build intelligent autonomous agents, are closer to success than anyone imagined we would be in 2023 (0-1 breakthroughs away, I'd say, for breakthroughs of the size we see once every five years or so), and publicly has no solution to the "don't share our interest" issue.
"Is this likely to harm society in anyway?"
"Uh, no. This is some messages and pictures on a website. Look how much money we're making. It's good for society"
<15 years of societal harm later>
"Yeah, we never thought this could be harnessed to spread misinformation, lies, unachievable standards of beauty and lifestyle, lead to surveillance and manipulation, or give rise to influencers and hussle culture..."
It's going to be exactly the same with AI. "Oh, it doesn't look like we created Skynet yet. No missiles in the air. Look at how much money we're making. It's good for society"
<15 years of societal harm later>
"Business Insider"
stopped reading
But where's the team protecting us from a future where we have no more shared experience or discussion of media anymore because everyone is watching their own personalized X-Men vs. White Walkers AI fanfic?
You keep using the C word without understanding the ramifications that it's not needed at all.
A virus doesn't consciously kill you. It's the closest thing we have to computer code in the wild really. It gets in your body and executes again and again until everything falls apart.
As a human, one you probably don't have the skill and ability to make viruses, and two, people that do are quite enamored with the ethical considerations because they are human and will be affected by it.
As a smart robot application making and testing viruses there are no self ethical considerations. You (the AI entity) would not be affected by your own creations. 'You' may not even consider ethics at all. Instead you pump to trillions and trillions of different strings and see what they do, until suddenly one day the meat puppets that tap your keyboards stop showing up.
Consciousness and sentience are just the things that would be novel about this situation, if they ever happened.
The 'if they ever happen' part is just a thought terminating cliche. Of course I do have to say that is "In my opinion", neither of us have the revelation of hindsight in this case. But, I don't believe humans are magical in any way. By the random walk of evolution a self perpetuating creature and is also intelligent was formed. It certainly seems it should be possible without a trillion^trillion tries to make it happen in another substrate. And I also find it odd by this random walk evolution found the only means to reach intelligence, and in doing so the most optimal form. Life has to self perpetuate within the confines of its own machinery. Raw intelligence isn't a carbon based lifeform and shouldn't have the same limits.
I don't think humans are magical in a spiritual sense, but for all we understand ourselves we may as well be, at least for the purpose of recreating consciousness.
The concerns that are more mundane extensions of the ways tech is already shitting up our lives are just so much more real and immediate.
Buyer beware :)
AI is and will continue to be developed for the purposes of war. Moreso the growth of the capability of AI could lead to destabilization and the occurrence of nuclear war.
For example the capability of MADD working keeps things somewhat stable "I kill you, you kill me" isn't great. But if I develop AI that could track all the sub, and knock out the nukes before they launch, this could push nation states to attack first before their capability is taken out.
There is a continuum of issues here spanning from "People problems -> people + AI problems -> AI problems"
By adding more AI we're just expanding the size of the problem space.
Maybe the "they're made of meat" phase is the great filter. Why would aliens speak with meat based organisms.
And how reasonable is that we would have autonomous agents a bit smarter than humans in the nearish future? Very low, I'd say, about the same as the odds I'd give a year ago for GPT4? Not odds I'd be happy to take for the _destruction of humanity_.
Can someone here identify with it?
Why not shave off those few curly hairs on his forehead?
Why wear a suit? Sitting in a suit next to the CEO of your company, who wears a sweater - isn't that giving off a very strange message? Like "I have to play by the rules, as I am not as cool and important as him"? Together with that golden smartwatch, isn't that giving of a "try hard" vibe?
This is not a critique. I'm just trying to understand. Is this a cultural thing maybe? Is this style fashionable or respected in some cultures?
This comment is subtractive from the discussion and doesn't belong here. It's just random insults about someone's appearance.
In the current Silicon Valley atmosphere of hoodies and sneakers, wearing a suit probably counts as a subversive act.