This just underscores the feeling that most of the problems people have with AI are actually problems with rampant capitalism. Negative externalities, regulatory capture, the tragedy of the commons -- AI merely intensifies them.

I've heard it said that corporations are in many ways the forerunners of the singularity, able to act with superhuman effectiveness and marshall resources on a world-altering scale in pursuit of goals that don't necessarily align with societal welfare. This paper paints a disturbing picture of what it might look like if those paperclip (profit) maximizing systems become fully untethered from human concerns.

I was always a little skeptical of the SkyNet model of AI risk, thinking the danger was more in giving an AI owner class unchecked power and no need to care about the wants or needs of a disempowered labor class (see Swanwick's "Radiant Doors" for an unsettling portrayal of this). But this scenario, where capitalism takes on a mind of its own and becomes autonomous and even predatory, feels even bleaker. It reminds me of Nick Land's dark visions of the posthuman technological future (which he's weirdly eager to see, for some reason).