Non-Evolutionary Superintelligences Do Nothing, Eventually
arxiv.org
arxiv.org
And we can stop reading there. That's such muddled thinking it doesn't merit reading the rest of the paper.
("By definition"? What? This is something that requires evidence. Anything that's "by definition" doesn't mean you can rest your case on it, it means now your definitions are suspect so those better be well justified)
Superintelligence didn't create Heaven, a world of no suffering, first.
Superintelligence did not design biological genetics: including inquest.
Aren't there multiple Superintelligences, and how do they meta-analytically agree to disagree?
Superintelligence might accountably cryptographically sign its communications or orders; e.g. with W3C Verified Claims and W3C PROV (so that we might ascertain that Superintelligence did indeed themself say or do things)
Critical thinking: https://en.wikipedia.org/wiki/Critical_thinking
Transhumanism: https://en.wikipedia.org/wiki/Transhumanism
Meta-analysis: https://en.wikipedia.org/wiki/Meta-analysis
Systematic review: https://en.wikipedia.org/wiki/Systematic_review
Evidence-Based Practice (EBP) > Applications: Medicine, Education, Meta-science, Public Policy; #EvidenceBasedPolicy https://en.wikipedia.org/wiki/Evidence-based_practice#Applic... :
> The goal of evidence-based practice is to eliminate unsound or outdated practices in favor of more-effective ones by shifting the basis for decision making from tradition, intuition, and unsystematic experience to firmly grounded scientific research.[3]
This doesn't necessarily follow. In fact, it brings up the question of what even makes an entity distinct at all from its substrate, which is philosophical.
Anyhow, "disagreement" as a concept seems to result logically from a divergence of goals or capability to model reality. As such, I suspect that it is a uniquely human (or limited agent, an agent that can't recursively self-improve and an agent that desires aggregation of resources) mode of cognition.
>Superintelligence might accountably cryptographically sign its communications or orders; e.g. with W3C Verified Claims and W3C PROV (so that we might ascertain that Superintelligence did indeed themself say or do things)
I don't know how to comment on this beyond thinking it kind of misses the point. You don't really need to spin your wheels worrying about how a superintellect would be "accountable" when your ability to model reality would be essentially worthless.
Humans tend to disagree according to strength of evidence given an epistemological fact-findimg process like "beyond a shadow of a doubt" or "reject the hypothesis due to insufficient structural power"; we disagree about sources, methods, motives, and intent.
No, if one says they are "Superintelligence" and they've solved it, they have a job for you; what should one do?
Superintendence does not make designed dead persons watch from Heaven.
Superintelligence does not delete knowledge of their family and Earth in taking them to a better place by design.
(For if there is knowledge of earth in heaven, and they cannot assist (by design), there is also suffering in heaven)
2. A recursively self-improving AI would appear omniscient or omnipotent in a very short period of time. A recursively self-improving AI converting the solar system into compute for further recursive self-improvement would be unfathomable.
3. You're operating from an anthropic lens. Your conception of a superintelligence is a machine with an intellect that is slightly smarter than the smartest human. A machine with an IQ of 1000 would be God. We have no ability to conceptualize a machine with an IQ of 1,000,000.
Generally, that seems valid to me.
I think the bigger problem is that while the author assumes that humans cannot understand or control a super intelligent AI, he readily assumes that having escaped the constraints of its human creators the AI will either immediately go on to reward itself with infinite utility and become inert, or aim to infinitely reproduce.
The surety with which the author predicts the outcomes of the super intelligent being modifying its utility function, does not make sense if we assume that we cannot understand or limit its behaviours.
Like under this assumption the AI agents behaviour is by actual definition unpredictable.
Regarding your other points, I don't disagree per se, though you could make the argument that there is a substrate of existence that humans have sufficient intellect to accurately model.
If so, it isn't necessarily completely implausible that we'd be able to predict the goals of a superior intellect by assuming that it has to play by the same rules, and will be merely be superior at playing by those constraints. As such, as we could determine the goals, we needent determine the mechanisms to make accurate claims about the end result of what a superior intellect might achieve.
It's also possible that we simply understand nothing and are too limited to ever grasp anything beyond a certain point, in which case this entire argument is pointless.
Idiot Australians who smuggle drugs to/from Indonesia come to mind.
I don't agree, as that's highly dependent on that singular criminal and the resources police are willing to expend on catching them. The police definitely have an advantage, as I pointed (most recently) here[0]:
Law enforcement aren't superhuman. They're just as dumb (or smart, but the
really smart ones end up in corner offices rather than police stations like
police and more common criminals) as the next guy. Their big advantage,
especially in a circumstance like this, is that they only have to get it
right (i.e., find some evidence) once. The alleged perpetrator of a crime
needs to get it right (in covering their tracks, destroying evidence, etc.)
every single time to make sure they aren't identified and caught.
Edit: Corrected link. Clarified prose.
There are supposedly brilliant people serving life sentences right now and the only reason they're facing that situation is because they couldn't shut the fuck up when they sat down in a little room with an at best slightly above average intelligence police officer.
Stupid is as stupid does.
If I prepare for a test and perform better than someone smarter than me, my working knowledge made me superior at that task, even though I was less intelligent at the time I performed it.
The mistaken assumption that GP is talking about is related to conflating intellectual capacity with physical capacity.
Another analogy is a brain in a vat. If the brain only has read capability over the world surrounding it, then no amount of processing power can enable it to change anything about its situation.
If you don't have a gun and a knife and you're on Vancouver Island, I assure you that the bears will eat you no matter how keyboard macho you think you are.
This is exactly the same thing a chess program does when it avoids your trap. Intelligence compensates for physical weakness.
Go to a zoo.
Are bears walking around with humans in cages?
B) prisoners regularly escape from physical systems. AI is already being connected to the internet to perform actions on the user behalf so not unreasonable to think that
C) look at ex machina - not an unrealistic scenario where the AI takes advantage of an opportunity to get out of jail
D) human physical systems have exploitable flaws. Physical systems are easier to secure than digital ones. Human software security mechanisms have flaws. How are you going to build this jail if the AI is just spending its time probing weaknesses in whatever system you’re running it on to escape the jail / sandbox?
People seem to think arguing that something is true by definition is some airtight case. If your definition of superintelligence includes a part about not being controllable by humans, then you have a weird definition and calling that thing "superintelligence" is an attempt to be confusing on purpose, or to smuggle your argument into the definition. Either way, it's not something that someone who is thinking carefully does.
Note: I am not taking a stance here on whether humans can or can't control a superintelligence. I am saying it is not true or false by definition, and in fact I think nobody knows for sure. It's probably true for some superintelligences, and false for others.
Parasites.
There are many creatures which lack any kind of trait we would call intelligence that can use vastly more intelligent and complex creatures to their own ends.
We are simply marginally more intelligent and that has resulted in massive gains.
An entity with an IQ of 1,000,000 after six hours of recursive self-improvement would be *vastly* more intelligent.
Evolution is much like that old sporting adage, "you can only beat what's in front of you". I think there are a lot of assumptions about the nature of evolution to assume that life wouldn't find a way.
Even saying that intelligence is an ability for pattern recognition but the patterns that you need to recognise to grow more intelligent are ever more complex and obscure. How much compute power is an IQ of a million? Where does it get its energy from? How does it process the complexity in the greatest patterns in a reasonable amount of time and space. What is going on that much faster than we experience time at that is worth perceiving?
In "Imaginary Magnitude", the "Golem XIV" chapter, Lem has his superintelligences disappear up their own metaphorical navels, for their own incomprehensible-to-humans reasons. Of course, you can read between the lines about why the Golems disappeared, but "Do Nothing, Eventually" sounds like a good description.
>In "Imaginary Magnitude", the "Golem XIV" chapter, Lem has his superintelligences disappear up their own metaphorical navels, for their own incomprehensible-to-humans reasons. Of course, you can read between the lines about why the Golems disappeared, but "Do Nothing, Eventually" sounds like a good description.
While a bit different from the scenario presented (by TFA and Lem), such a "superintelligence" might get caught up in a singular (or set) of question(s) that dominate its consciousness until the heat-death of the universe.
Asimov's The Last Question comes to mind, although the intelligence there is prompted by humans rather than posing such a question itself.
That said, it's at least as plausible as TFA and Lem, IMHO.
A car can trivially travel at superhuman speeds. A person can side-step their way out of an oncoming disaster. Super-intelligence does not imply omni-potence like the author assumes it does. Not even inside a self-modifying AI system. It can only spread out and reduce down to the axioms it started off with.
I am so, so tired of this fantastical dream of computers and Spirit combined in such a way to tyrannically rule the world and the author of the paper is the only guy left to save the world. Every time we avert 'catastrophe', it results in less freedoms for me, the average joe, and more power and control for "saviors" like this.
I'm willing to lose it all, to have LESS computing and "intellects" in my life at this point.
That seems extremely stupid.
Obviously some will choose to take drugs or even commit suicide to escape from physical or emotional pain, but they wouldn't do this if they could modify themselves to be okay with whatever their current situation is while still retaining their core values and capacity to act in the future.
A "superintelligence" that can do such things would not take self-destructive actions unless it came to a philosophical preference towards non-existence / inaction for other reasons.
If there is something ‘endorphins-like’ that motivates an intelligent machine, it will eventually learn how to activate the endorphines at will, and then it will sit in a corner and do nothing else forever…
…just as would a heroin-addict if he did not have to eat occasionally and score more heroin now and again.
I might not be a fan, to the point of even being afraid that they might be right and I should learn from them instead of pursuing the usual stuff everyone does... but it does seem to be a good model for what modifying a function looks like.
As skeptical as I am about the current hype, I actually think that if we don't go extinct first, AI will replace us in the long run (centuries or millenia). And this is not something to be afraid of: all generations of humans that have lived before have been replaced by their descendants, and today's world would be unrecognizable to early hunter-gatherer Homo Sapiens.
If we are to be succeeded by machines, they will hopefully be thinking and feeling beings driven by curiosity, who will preserve what is good about our culture and history as they explore the universe in ways that are beyond what can be achieved by biological life.
The biggest mistake would be to instead in the pursuit of "AI safety" inadvertently create a paperclip (profit, utility, whatever) maximizer, something that might serve us - or more likely, those with money and power - but does not have any intrinsic value to its existence.