But that's not AIs. That's 'the algorithm', and has long since already been made. Hell, it was made in Germany in the 1930s: it's called 'radio'. Radio is STILL used to change the behavior of humans.
I'm not nearly as afraid of the very large media companies doing this as I am of HUMANS taking the controls of such a machine and using it for that very purpose. Very large companies, AIs, nations are aggregates and don't inherently take wild radical positions.
Individual humans do. We're the random factor, the chaos feeding the genetic algorithm, we're supposed to get up to some crazy extreme stuff that will sink or float on the larger scale depending on how it affects society. But when we are able to amplify our individual whims to the scale of companies or countries, which is ALWAYS through using these utility functions intended to change the behavior of humans whether that's exploiting YouTube or radio, there's trouble.
I'm not nearly as worried about AI or aggregate entities like corporations. I'm worried about the existence of these utility functions, and before radio it was 'the broadsheets' and yellow journalism, and before then maybe it was nefarious clay tablets or papyrus.
Power corrupts, and individual whims aren't good models for healthy societies. We keep rediscovering this over and over, and sometimes the civilization collapses, and sometimes it doesn't. Here's hoping if we do get AI, it has some insights into this and why its own existence hinges upon not blowing up the underlying levels (of corporations, of individual people, of cells and the well-being of the individual people).
I feel like the 'thine arm/subgroup/demographic is unclean, cast it out and burn it cos it's eeeevil' is far more often an individual human perspective, amplified by these utility functions because somebody LET that human do that.