I'm kind of upset to see systematically "Alignment" and "AI Safety" co-opted for "undesirable business outcomes".
These are existential problems, not mild profit blockers. Its almost like the goals of humanity and these companies are misaligned.
These are existential problems, not mild profit blockers. Its almost like the goals of humanity and these companies are misaligned.
> Its almost like the goals of humanity and these companies are misaligned.
A well functioning market has competition that not only drives down prices but increases quality of products. Obviously customers want to simultaneously maximize quality and minimize price. But when customers cannot differentiate quality, they can only minimize price. Leading to the feedback loop, where producers are in a race to the bottom, making sacrifices to quality in favor of driving down prices (and thus driving up profits). Not because this is actually the thing that customers want! But because the market is inefficient.
I think critical to these alignment issues is that they're not primarily driven by people trying to be malicious nor deceptive. They are more often driven by being short sighted and overlooking subtle nuances. They don't happen all at once, but instead slowly creep, making them more difficult to detect. It's like good horror: you might know something is wrong, but by the time you put it all together you're dead. It isn't because anyone is dumb or doing anything evil, but because maintaining alignment is difficult and mistakes are easy.
This is all much much less of an existential threat than, say, nuclear-armed countries getting into military conflicts, or overworked grad students having lab accidents with pathogen research. Maybe it's as dangerous as the printing press and the wars that that caused?
But machines? Well they have none of that. They're optimized to make errors difficult to detect. They're optimized to trick you, even as reported by OpenAI[0]. It is a much greater existential threat than the overworked grad student because I can at least observe them getting flustered, making mistakes, and have much more warning like by the very nature of over working them. You can see it on their face. But the machine? It'll happily chug along.
Have you never written a program that ends up doing something you didn't intend it to?
Have you never dropped tables? Deleted files? Destroyed things you never intended to?
The machine doesn't second guess you, it just says "okay :)"
[0] https://cdn.openai.com/pdf/34f2ada6-870f-4c26-9790-fd8def563...