> "This work is part of the third pillar of our approach to alignment research: we want to automate the alignment research work itself."
I feel like this isn't a Yud-approved approach to AI alignment.
I feel like this isn't a Yud-approved approach to AI alignment.
Ie, I think it's not that this shouldn't be done. This should certainly be done. It's just that so many more things than it should be done before we move forward.
Your comment triggered a random thought: A perfect name for Yudkowsky et al and the AGI doomers is... wait for it... the Yuddites :)
https://en.wikipedia.org/wiki/Eliezer_Yudkowsky https://twitter.com/ESYudkowsky https://www.youtube.com/watch?v=AaTRHFaaPG8 (Lex Fridman Interview)
I'm willing to bet the future of our species on my consistent victory in these types of matches, in fact.
In any case a similar argument can be made with merely instrumental goals causing harm: “I am an ant and I do not see how or why a human would cause me harm, therefore I am not in danger.”
Honestly if you have no examples you can't really blame people for not being scared. I have no reason to think this ant-human relationship is analogous.
And seriously, I've made no claims that AI is benign so please stop characterizing my claims thusly. The question is simple, give me a single hypothetical example of how an AI will destroy humanity?
The guy at Google already demonstrated that AIs are able to convince people of fairly radical beliefs (and we have proof that even humans a thousand years ago were capable of creating belief systems that cause people to blow themselves up and kill thousands of innocent people).
P.S. I was not characterizing your opinion, I was speaking in the voice of an ant.
Other caveman use anthrax in subway station. Anthrax scary and hurt…
Is AI closer to fire or closer to nukes and engineered viruses? Has fire ever invented a new weapon system?
By the way: we have shitloads of regulations and safety systems around fire due to, you guessed it, the amount of harm it can do by accident.
Take this blog post for example, which between the lines reads: we don't expect to be able to align these systems ourselves, so instead we're hoping these systems are able to align each other.
Consider me not-very-soothed.
FWIW, there are plenty of AI experts who have been raising alarms as well. Hinton and Christiano, for example.
And what does charisma of AI alignment folks have to do with anything?
Nuclear weapons proliferated explicitly because they proved their scariness.
We have figured out stuff in the past, but we also came shockingly close to nuclear armageddon more than once.
I'm not sure I want to roll the dice again.
Anyway this is a good example of the completely blind-faith reasoning that backs AI optimism: we’ll figure it out “because it’s what we do.”
FWIW we have still not figured out how to dramatically reduce nuclear risk. We’re here just living with it every single day still, and with AI we’re likely stepping onto another tightrope that we and all future generations have to walk flawlessly.
I'm not sure Yudkowski is an EA, but the EAs want him in their polycule.