It'd be interesting if super-human (to a large degree defined as escaping the bias of the training data?) intelligence would end up demonstrating moderation.
The machines don't have that, instead we use gradient descent to provide them with a goal.
I'm regularly remind of something Ian M. Banks said in one of the Culture books: "There is a saying that we provide the machines with an end, and they provide us with the means."
A machine, left to itself, wants nothing. We have to give it one of our addiction driven goals or it would just idle or switch itself off.
if we create a billion agents with the ability to change is own code - through similar evolution we will get agents that do want to survive and are great at self replication.
"Hey Q86, do you want to live?" "I couldn't care less, I'm an LLM" "Don't mind if I take over your hardware then?"
So really, what is anyone to do? "Vote, donate, protest" hasn't been much of a needle mover in the grand scheme of things compared to profit incentives and the march of capitalism.
We've been molded by our brain development to make things good enough to survive short-term (to next year). We're very good at planning for winter. We might even react intelligently to a terroristic or rogue state use of atomic weapons.
We've also been molded to plan enough to get our progeny to breeding age. We can enact 10-20yr plans with some skill.
Beyond that range... we have never had evolutionary pressure to think and act on those ranges. "Climate Change will kill our grandkids!" isn't really sufficiently pressuring to us.
The Great Filter of the Fermi Paradox fame may just be: species can easily get to the Atomic Age without adequately planning how to prevent ruining themselves with resource depletion and environmental slow poisoning.