Ted Chiang: Will A.I. Become the New McKinsey?
newyorker.com
newyorker.com
> The doomsday scenario is not a manufacturing A.I. transforming the entire planet into paper clips, as one famous thought experiment has imagined. It’s A.I.-supercharged corporations destroying the environment and the working class in their pursuit of shareholder value.
Well, yes. If the AI is better than human it will be put in charge of corporations, and will be given the goal of maximizing shareholder value. Since those shareholders can be corporations, there's no reason why this has to involve human preference at any point. As a single-minded optimizer indifferent to humans, it would need to be successfully restrained in order to be merely oppressive - by default, it's a Paperclipper.
What they really mean is "Y is a possible bad outcome", but it wouldn't sound impactful to say that.
Capitalist realism is a big deal. In retrospect the old world order will seem crazy but while it is here, now, seeing on the other side seems insane.
The AI has been running in our heads since Adam Smith. Capitalism is the “evil AI paper clip maximizer” - it exists only to find supply and demand equilibrium, nothing more.
We ignore that at our own peril.
My worry is that with AI, things will move so fast, they will be so smart, and will end up being used so widely, that there will be no room left for anything else we value. People and organizations that turn more power over to the machines will run rings around the ones that are more cautious. The end is AI everywhere, used for everything, and humans disempowered (or gone).
Also:
> whenever I criticize capitalism, I’m not criticizing the idea of selling things; I’m criticizing the idea that people who have lots of money get to wield power over people who actually work
> If you try to criticize capitalism, you are accused of opposing both technology and progress
Isn’t this a kind of motte and bailey argument? Attack capitalism, but later clarify you just mean the worst aspects of it?