The Misalignment Museum
misalignmentmuseum.com
misalignmentmuseum.com
Well, we don't need AI to do that. We're doing pretty well all on our own, thank you.
The popularity of libertarianism with them also seems distinctly in favor of power hierarchies rather than the libertarian communist types. So when these viewpoints stoke fear I wonder what what kinds of solutions they are driving at
Although, with a few conversations that I have had with the "alignment" enthusiasts gave me the impression that in their opinion, many of the hot button issues like climate change/social inequality/global poverty etc. are irrelevant and not worth spending time over, because AI alignment is a much more serious threat. In that view, 90% or more of their political position overlaps with the conservative crowd ("issue X/Y/Z is not worth our time, and we should just let it happen as it does now"). So I am not surprised an outsider may label them as conservative.
Capital-unism sounds awful enough
Not following the logic at all here.
They are describing their own view _and_ attributing their reasoning to others.
But the argument isn't valid.
So if you have liberal values and consider a hard takeoff a likely near-to-medium-term scenario, then it's perfectly reasonable to argue that you should prioritize alignment research over almost anything else, because it's a critical instrumental goal for achieving all your other goals. This by no means makes you any less of a liberal, or more of a conservative.
But people who are more interested in virtue signalling than results, and/or don't know about or believe in a hard takeoff scenario, may interpret this train of thought as "conservative" because it appears to not match the standard thought patterns that a good, card-carrying liberal is supposed to have.
In all LW surveys to date, people who have identified as "conservative" or "reactionary" have been a tiny minority. There are many libertarians (whether "l" or "L" or whatever libertarianism means to different people), but the "liberal/social democrat" block is the largest. With regard to American party affiliation, the large majority lean Democrat. Most are atheist and lean consequentialist ethics, neither being a sign of conservative or reactionist values.
Anyway, the rationalist community was never about people who were particularly rational, it was all about people recognizing their own irrationality and biases that inescapably comes with being equipped with a human mind. Of course, that does not mean it didn't attract people who thought it was about being a tribe of us, the rational and enlightened people vs them, the irrational plebs. Which is rather ironic, of course.
the liberals among them or capitalist libertarians don't round out the spectrum of political views when it comes to ideas that community popularly advocates.
advocates for strong gatekeeping and violence against any public subversion of it, within a centralized power whether capital or state, is getting at different flavors of the same kind of tyranny that you'll end up with whether liberal, conservative, capitalist libertarian.
But the agenda here is very simple: make sure AI doesn't kill us.
They believe that earnestly. That is the end goal. Everything is in service of that not the other way around. And with how quickly AI is progressing I'm pretty onboard with that.
"The agenda of Christianity here is very simple: make sure the devil doesn't kill us. Christians believe that earnestly. That's the end goal. Everything is in service of that and not the other way around. And with how quickly the world is turning to shit, I'm pretty onboard with that."
There's a blind devotion to a group's chosen x-risk event, alongside assigning infinite utility to that one goal at the exclusion of all other useful goals. That's an important hallmark of fundamentalism. I sincerely don't think it's a good thing.
You have faith that AGI will both exist and have similar power. That's your genuine belief, you've been convinced by the evidence that you see. Maybe you're not 100% there but that's okay - you plan for this low-probability but important event by reading up on machine learning, contributing to your community discussions, studying alignment, performing outreach, converting folks who don't believe in AGI, pretending to read Norvig textbook, etc.
My point is that these two beliefs are related.
Yes.
> Neither has been directly observed, after all.
Exactly, they are both just ideas people have that help make sense of the world as they see it.
Is there any low-probability but important event that doesn't follow this template?
I don't believe this to be true. Nor do I believe the AI research community broadly believes this. Other goals can certainly coexist.
But when the median AI researcher believes there's a 10% (!) chance that what they're working on will destroy the world, and the pace of that work is only quickening, that seems like a big issue to worry about.
To quote casebash elsewhere https://news.ycombinator.com/item?id=35195102
> Just because people decide to focus their time on one issue (say AI), doesn't mean that they don't appreciate people working on other issues.
Yet they will not quit their job. What that suggests to me is that they don't really believe it, otherwise they would sacrifice their paycheck to stop it. Seems to me that what they really believe is that they should get more attention and more funding.
I've talked to quite a few researchers. They generally do earnestly believe that AI has huge damage potential. The answers to why they keep developing AI falls in several different buckets (to which I have varying degrees of agreement)
1. We're in a prisoner's dilemma. If everyone could unilaterally agree to slow down AI research I'd be happy to quit, but until that happens if I don't do it someone else will and with even fewer scruples.
2. The only way to develop AGI safely is to push things to the brink. It's impossible to make progress on how to make sure that AGI is safe is to almost make AGI because not until then do you have a clear enough picture of the exact form AGI will take to do productive safety research.
3. Slowing down AI research has its own risks. If you slow down AI research, but other fields of technology continue to progress, you can end up in a scenario where everyone's personal computer is capable not only of running models, but of doing cutting-edge ML training. Right now to coordinate on developing AGI safely, there are maybe ~10 top teams with the infrastructure possible to do the training (not just running the model!) needed to advance the state of the art you have to get cooperation among. Hard, but not impossible. If technology progresses to the point where there tens of thousands of teams all capable of meaningfully advancing the state of the art then coordination is absolutely impossible.
4. If AI can, then AI should replace humans entirely. (the most chilling response I've heard. Luckily few in number, but it shows up even on HN)
The short of it is yes. As far as I'm aware the long-term hope in the Less Wrong community or more generally the AI safety community is that AI becomes available for everyone and under the control of everyone.
But the first, paramount step is to make sure it doesn't destroy everything and recognize that such a guarantee doesn't come for free, but rather requires an enormous amount of work and coordination.
I don't take lightly to suggestions that we enact police states, highly authoritarian government controls, and other state or capital extremes that I've seen that community / the alignment folks advocate for as you can find when you look past their concern and fears
They don't really have an answer, which is why they're freaking out.
> I don't take lightly to suggestions that we enact police states, highly authoritarian government controls, and other state or capital extremes that I've seen that community / the alignment folks advocate for as you can find when you look past their concern and fears
What comments are you thinking of? And what solution would you advocate for in a scenario where creating superhuman AGI that kills everyone is much easier (and probably the default) than creating superhuman AGI that doesn't (basically how do you avoid prisoner's dilemma-esque scenarios given how much power AGI gives its makers right before it kills them)?