Can a group of Chimpanzees build a cage that can contain a Human?
It's not at all obvious that a super AI would have an intellect incomprehensible to a human the same way a human is to a chimpanzee.
Or another way to phrase it, it's not at all obvious that an AI can exist that would be incomprehensible to a very smart human, it may however reason much faster than such a human.
I think similarly, a super AI's intelligence will be a completely different type of intelligence than what we have. For the lack of better word, it exists in a different dimension.
For example, things that are blackbox to us will be completely comprehensible to that super AI. Or it can solve problems that we might have considered impossible to solve.
This is nonsense, magical thinking. It's possible to model reasoning formally; it's called "logic". A system of deductions built upon some axioms. Any logical reasoning, no matter how complex, can be expressed in such a system, and can be understood by anyone else given enough time.
Humans approach large complex proofs with symmetry arguments and case splits.
These are not necessarily going to be universal in all kinds of reasoning.
Computers are perfect logicians by default, but AFAIK no logic compact enough to be human-comprehensible has been enough to see, hear, or read. At least, not reliably so.
Logic gates are combined to form binary numbers, which are used to label symbols and approximate reals, upon which calculus is approximated in toy models of neurons, which are combined and trained and eventually learn to add numbers and then puts the wrong number of hands onto the third arm of the human it was tasked with drawing.
It definitely has "diminishing returns to acquiring resources". There are people many IQ points higher than Musk but none of them are anywhere near as wealthy as him, and it's not clear that Musk would be richer if he was smarter.
I imagine such advantages also come with at least a lower time bound on solving many classes of problems and presumably that would be experienced as an incomprehensibly smart intelligence. I imagine it'd feel like chess computers in most areas of life in that the AIs actions would feel impossibly perfect at all turns leaving us far behind in attempts to compete
An AI mind which can learn and intuit as well as an IQ 130 human (2σ, the tests become unreliable above that), but also comes with the speed difference between synapses and transistors (roughly the same as the difference between jogging and continental drift), has a chance to become expert at every subject.
Most of us have enough difficulty truly comprehending the domains of other single human experts; a human-upload with that much breadth of expertise will be incomprehensible by default even if they speak your language.
https://www.treehugger.com/chimpanzees-endangered-5220730
Their population is only a third from 20 years ago:
> The Jane Goodall Foundation estimates there are between 172,000 and 300,000 chimpanzees left in the wild, a far cry from the one million that existed at the turn of the century.
> Poaching and habitat loss due to illegal logging, development, and mining continue to plague wild chimpanzees in their native habitats across Central and West Africa. These issues lead to other indirect threats, such as diseases due to increased contact with humans.
> Chimpanzees are more commonly hunted using guns or snares, while poachers often target new mothers in order to sell the adult as bushmeat and the babies as pets.
This is why I hate analogies.
Hitchcock's The Birds but its chimps. Sounds like a job for Stable Diffusion...
This is also why analogies like this are not useful, these aren't comparable qualities in the subject the analogy is talking about.
Any AI that is intelligent enough that needs containment can’t be contained by humans.
An AI which is just about intelligent enough to need containment, and can be contained only by the smartest N humans.
At some point you're going to need to exit...