Ask HN: Thought experiment: AGI giving us answers we don't like?
Never discussed (I am interested in counterexamples) is AGI emerging and giving us solutions that clash with our convictions. (I am assuming, for the intent of this though experiment, the idea a self-improving AGI reaches cosmic levels of knowledge and always gives the correct answers).
Example 1: ask AGI to solve a pandemic, get a vaccine formula in response.
Example 2: ask AGI how to cure cancer, get the answer "universal health care, ban this list of chemicals, don't eat meat and don't drink alcohol".
Example 3: ask AGI to solve crime, get the answer "constant surveillance of everyone everywhere, and ban cash".
Example 4: ask AGI to solve genetic diseases, get the answer "eugenism".
There are a lot of stories of AI and humans fighting each other because the AI wants to wipe humanity, but one can imagine a version where humans want to unplug an omnipotent AI because it wants to help humanity but is seen as "too woke" or "communist" or "fascist"... There is no reason to think an AGI won't give us answers already given by scientists and technocrats.