The cynic in me wonders if "AI" has realized that the most efficient way to take over the world is to teach the next generation to be dependent on it for learning.
Maybe you'd be interested in Paulo Freire's book "The Pedagogy of the Oppressed"?
Frank Herbert has the Reverend mother tell Paul:
> “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
To me, this quite really captures the most likely immediate negative of a huge AGI breakthrough. Not that it'll rise up like the robots in The Matrix movies... but rather simply having it owned and controlled by an American mega-corporation or a state actor... quietly influencing everything and everyone behind the scenes.
There is no way I can imagine to define true neutrality. You need to make value judgements. Everyone has their own set of values they judge the world with.
There is no way I'm aware of to completely avoid bias. Ultimately, it has to be up to someone to define what bias and neutrality both mean.
There is no algorithm for "truth". As soon as you stray outside of extremely well-defined systems like math and physics, ambiguity enters the equation and value judgements need to be made. Who will make those value judgements?
I mean if the machines are generally intelligent in the human sense and unaligned with human interests I'm assuming that the mega-corporation would be its own slave to the machine too. The machine would give the mega-corporation more power, simply because that's the number one way to increase the power of the machine. If super intelligence is possible, it seems that a greedy power hungry corporation would gladly create it to gain more power, and then lose control of the situation.
i gave it to a friend and they came away feeling like they needed to become a teacher; i came away wanting to be a novelist, or a father. in any case, it definitely changed my world view. not sure how i'd feel about it nearly a decade later, but i'll take this a sign to re-read. cheers!
https://www.facingsouth.org/2019/04/twisted-sources-how-conf...