Until the alignment movement begins to take seriously the idea that we already have misaligned artificial general intelligences I think they are best viewed as a convenient foil
Paperclip maximizers exist, they're made not only of code but of people
Paperclip maximizers exist, they're made not only of code but of people
https://ai.objectives.institute/whitepaper
It’s weird to have been working on a paper for almost a year and have it launch into this environment, but uptake has been good. My hope is that we will continue to see more nuance around different kinds of alignment risks in the near future. There’s a wide spectrum between biased statistical models and paperclip maximizing overlords, and lots bad but not existentially catastrophic things for the public to want to keep a pulse on.
> In some sense, we’re already living in a world of misaligned optimizers
I understand this is an academic paper given to nuance and understatement, but for any drive-by readers, this is true in an extremely literal sense, with very real consequences.
In this sense, I'm pleased to see Open AI claim to be taking a more careful stance, but to be honest I think the genie is already out of the bottle.
> Some epidemiologists are worrying that a new virus from Wuhan could become a wider catastrophe. Their message is infecting people around the world with fear and xenophobia, spreading faster than any plague. Perhaps they should consider that in some sense, they themselves are the global pandemic.
Like, yeah, people did consider that idea, and the "corporations are the real unaligned AI" idea, and the "capitalism is the real extinction risk" idea, and all the pseudo-clever variations of the concept.
The problem is that "understanding that capitalism has problems" isn't equivalent to "having an actionable plan to solve capitalism".
This is a caricature and the same thing could be said of AI x-risk. There are plenty of ideas on how to avoid unwanted effects of economic systems. I don't think it's at all clear that it's a single problem with a single solution. Getting ideas into practice tends to be the tougher challenge.
More broadly, the point is not to say "wow, alignment problems have existed for a long time already!" This is not profound or clever, it's obvious. But there's a big group of people considering a narrow definition of the problem, and playing what could be considered a useful social role.