Now imagine we are the worm.
Now imagine we are the worm.
This discussion centers too much on the definitions of words like superintelligent and reminds me a lot of philosophical discussion about omnipotence. Both seem to rely more on defining concepts first and then assuming their existence as a consequence.
Super-intelligence, not so much — there's plenty of examples of above-average humans in many areas, and no reason to think that putting the top expert in each field into one room is impossible, and no reason to think that this configuration cannot be implemented in software with a sufficiently powerful computer.
And that's without the things that machines already beat us at, because super-human chess playing software is easily available, and computers that do arithmetic better than the entire human species even if we were all trained to the level of the current world record holder are cheap enough to be given away free glued to the front of a magazine, so there's no single person who has a particular advantage with those things.
What chess does do, is give an example: if I was playing a game with Kasparov, I would have no idea which move he might make at any given moment, but despite that I'd still expect him to win.
With an AI, I don't even necessarily know what "game" it's playing even if I'm the one who wrote the utility function it's trying to maximise.
God could make a burrito too hot for himself to eat. Then he could make himself capable of eating it. Rinse and repeat.
Either way I'd say Matthew 4:1 solves that paradox in a way that satisfies billions of people. Though I am getting a kick out of imagining Mexican/American Jesus munching on burritos.
I don't understand why that would have any effect?
"Then Jesus was led by the Spirit into the wilderness to be tempted by the devil."
First, if you want to sidestep if Jesus is omnipotent, then it's no longer saying anything about omnipotence.
Second, you also have to make the devil omnipotent.
Actually, the second point has a sub-point, because if it's two different entities then you get "an unstoppable force meets an immovable object" which is more easily resolved by "the force just passes right through the object without influencing it" — for the self-contradiction I gave to even be present, you would need to regard the devil as being another aspect of Yahweh in the same regard that Jesus is an incarnation of Yahweh within Christianity (not to be confused with how Jesus is seen in Islam, which totally rejects this as polytheistic idolatry).
Can we? What constituent things would make up a superintelligence? Who's to say that our answer to that question is what is actually true in the case of a real superintelligence?
> Both seem to rely more on defining concepts first and then assuming their existence as a consequence.
Unlike religious philosophers like Anselm of Canterbury and Descartes and their ontological argument for the existence of a supreme being merely by imagining it, I don't believe anyone in the study of superintelligence is presupposing that they exist, or even can exist, they only presuppose how one might hypothetically exist.
It seems much more realistic to me that AI will be running systems that humans used to run, and therefore will understand at some level.
Now imagine we are that computer.
Humans are capable of abstract thought and reasoning, which is sufficient to understand even the most complex software programs, to such an extent as they are understandable. The comparison to the worm is bogus.
To actually match the worm analogy, imagine a clocked-down Turing machine with a tape length of maybe 100k cells. Now you can definitely imagine programs that can't run on it.
The worm's limitation isn't that it's slow at thinking, it basically has only a handful of neurons compared to a person and there are problems/concepts that are just too large to fit in its brain and it would never be able to imagine even if it could live and think until the end of time.
The basic argument here is specifically about the practical limitations. An actual worm, with a finite number of neurons and a finite lifetime, even if it was an exceptionally clever one, lived much longer than average, and spent every waking minute of its life pondering the mysteries of humans, would never be capable of comprehending more than the tiniest fraction. Even if you had worm scholars that developed some form of writing and could pass along information to subsequent generations, it's still not going to be possible because of the actual, physical limitations that have to be dealt with. A theoretical intelligence of a similar ratio to human intelligence would be similarly incomprehensible to a really clever human that spent their whole life studying the output of scholars who spent all their lives trying to understand it.
I’m aware of Bostrom’s argument and it is bogus. There isn’t a continuum of intelligence difference between a worm and a human, and a human and a so-called superintelligence. There is a categorical difference between the worm and the other two, which occupy the same class. Hierarchical abstract thought allows the human to reason (using cognitive artifacts like pencils or computers) about processes of any complexity, including those which vastly outstrip its own.
Boston’s a philosopher, not an engineer or computer scientist, and it shows when he makes basic errors like this.
No, you did when you brought in Turing machines:
> Imagine a clocked-down Turing computer, whose operating speed us is so slow that we can see its operations. Now imagine the programs which cannot, in principle be run on this computer. You can't because there aren't any.
If the Turing machine doesn't have infinite capacity, I 100% can imagine programs that "cannot, in principle be run on this computer".
Furthermore, if the Turing machine has many orders of magnitude less capacity than a computer and can only run for a tiny fraction of the number of cycles before it grinds to a halt, I can imagine many, many, many programs that won't be able to run on it.
> An AI isn't infinite either.
It doesn't have to be. Just being many orders of magnitude "larger" than us is enough that it, for all practical purposes, is incomprehensible to us enough that it might as well be an entirely separate category.
There are still no infinities.
Human beings have access to the same amount of storage and processing power as AIs, because we get to use computers too.
You brought up infinite machines, hence my and the other person's comment. Turing computers or machines have infinite tape. Maybe you meant to just say "computer," not "Turing computer."
Humans can use computers but only for human readable tasks. There is no guarantee that we could comprehend a superintelligence running off a supercomputer, we already don't understand neural network internals at even their current stage.
Maybe the norms in HN are different, but in my field a Turing machine is primarily a general purpose computer, and “Turing complete” describes models which can represent any Turing machine within its constraints. An “infinite tape Turing machine” is explicitly specified when needed.
Personally I’m a bit outside of the mainstream in that I never use infinities except in the case of representing an unterminated series. I reject that “infinity” as a number even makes sense as a context.
Bostrom is a charlatan playing rubes with an intellectual version of three-card monty:
//This paper argues that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed.//
What is a post-human stage of development?
Is human extinction a post-human stage of development?
How did this argument get from post-human to ancestor simulation? It's stipulated in a lemma.
What is a simulation according to Bostrum? Undefined.
What then is an "ancestor" simulation?!
The weaseling on display adds gross insult to intellectual injury:
How unlikely is "extremely unlikely"? What is a "significant number" of ancestor" simulations? What is "evolutionary history"? How did he get from ancestor simulation to computer simulation?
Who else can stake an academic reputation on tenets delivered in the form of afterthoughts?
A theory must contain a promise of making matters simpler. If there's any kind of theory associated with Bostrom's argument, it's that many people can't distinguish a theory from an arbitrary collection of intriguing statements. Bostrom is functioning as an academic instance of Weisenbaum's Eliza: it just circularly echoes its own kooky conjectures in the form of lemmas and an outwardly spiraling discussion of itself.
Bostrom is playing a silly joke on readers. He gives his game away with his ridiculous lemmmas, showing the ace during the shuffle:
"It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor‐simulations is false"—period.
So, what was the question which was to be begged? Read it and weep. You've been played.
You've got to hand it to a magician / mentalist who gets his audience to believe they are as "worms wiggling in the ground with no conception of god-like beings that inhabit the world".
The most pitiless aspect of this fantasy is that it includes a proposition for what man is unaware of in the form of the overt statement of that which he is unaware: "god-like beings" Could the question of faith be any further begged? It's deranged.
Here we encounter Zizek's fourth quadrant of Rumsfeld's knowns: the "unknown knowns." Ideology. That which you know but avoid or refuse to become aware that you know to keep your faith alive.
There's a rule about the actual unknown, which is that it must be literally about the which is not known!
More generally, as to what is not known, we can only poke at the edge of a lacunae, maybe inferring something about the contours or gradient to mystery, maybe following the edge to refining our adaptation to ourselves. But our work doesn't belong with the unknowable. It belongs with what we can know about ourselves. The unknown will take care of itself.
In terms of projects I'm working on, I'm traveling currently and it's a pain to track how much money I've spent due to needing to convert foreign currencies, so I'm building a simple app for that.