There's an xkcd comic titled "A Bunch of Rocks" at https://www.xkcd.com/505/. Could dropping rocks in the sand to simulate a turing machine produce a conscious being? If not, why can a computer?
You think this is some special matter? It's just carbon atoms all the way, which even started out as simple chemicals in a pond.
And just like you can't see "consciousness" in the brain (it's just a piece of meat with some electrical signals moving inside), you can't see it in the rocks either.
But the consciousness is in the encoding of the rock structure not each individual rock. Just like molecules taken out of the brain are not conscious, or the brain itself, when dead with no one to "move the rocks" (send neurons electric signals) is not conscious. It's the structure that is encoding the consciousness.
[Edited after reading the XKCD strip, in which by "throwing rocks on sand" means immitating a turing machine with rocks and sand].
Except if you think it's in an external, not meat/neurons/info soul.
Of course you can't test it empirically, because there's no empirical test for subjective experience. You can only test for the appearance of subjective experience - specifically for external behaviours, loosely defined, which might be considered to imply subjective experience to you as an observer.
This isn't a solvable problem, and Bostrom is begging the question by trying to hand wave it away with a sneaky "If we just accept this premise..."
The only reasoning I have seen on this subject so far has been "people are special" or "God made me". Sometimes the arguments are dressed up more. Do you have a good source that has a different type of reasoning?
Edit:
> You can only test for the appearance of subjective experience
Yes, and we solve this in people by assuming that if they sound, act and look like a person they are a person. I've never seen a compelling argument why it is somehow wrong to apply this principle to everything else, except of course the huge trouble philosophers go to to then argue what exactly needs to be looked at before we are satisfied. For some reason the Turing test is not adequate, but to my mind it's only not accurate if you believe that you are somehow special and that there is more to you than what can be observed.
Let's say I build a chatbot with a predetermined database of perfect responses to every possible string of words. This is quite clearly possible in theory, given a database with infinite room. This chatbot would pass the touring test, even though all it's doing is retrieving prewritten sentences. Is it conscious?
Ergo, it's quite clearly possible to build an automaton which acts like a human, but actually has no clue what it's saying. Which means the touring test isn't adequate.
---
Here's another take:
You have no proof that anyone in the universe is conscious other than yourself. However, given that you were born in the same way as everyone else, and you're conscious, everyone else probably is too. Why would you be so special?
Well, a computer program was not born in the same way that you were, so that assumption goes out the window.
For all intents and purposes, yes. Though without actual consciousness it would be impossible to build it.
>Well, a computer program was not born in the same way that you were, so that assumption goes out the window.
Well, a computer airflow program was also not created in the same way as actual air, but it can simulate it to the degree that we use it to build actual planes.
A computer program might not have been 'born' in the same way that we are, but we can make it simulate the "way that we are" (giving enough computer power, down to the atom or quantum element). So?
Care to mention any of those "good practical reasons"?
Penrose gave it a try (though even there, there was nothing about how it can't be modelled with code for example, in which case, rocks are fine too for a simulation a la XKCD, just not as a direct substrate for an Android).
>Of course you can't test it empirically, because there's no empirical test for subjective experience. You can only test for the appearance of subjective experience - specifically for external behaviours, loosely defined, which might be considered to imply subjective experience to you as an observer.
It's not really that we can't test it, it's more that we can't measure it. But we can, and do, test it, e.g. with psychological tests, the same kind we give to people to understand different aspects of their personality (which we also can't just measure with some equipment or simple measurement). If it quacks like a duck, and looks like a duck, and cries like a duck, and has other sentimental duck qualities, it's a duck. We only accept the subjective experience of other people from its appearance (their actions etc) too, anyway.
Given that meaty goo can be conscious, and assuming that this arises from physical properties rather than some form of actual magic, it is reasonable to guess that the consciousness arises from what the goo does rather than from other properties such as pinkness or gooiness, so if we can cause another physical system to do the same things then it’s likely to have the same subjective experiences while doing it.
https://www.reddit.com/r/MachineLearning/comments/2xcyrl/i_a...
> "...the agent could be called self-aware."
"Could be called" being the key phrase. Yes, you can call such a model self-aware, because indeed it incorporates a model of itself so it is self-aware in a very literal sense.
However, there's no reason to believe that a model with the ability to represent itself would have conscious experience of the kind humans do. Why should we expect it to be any different from any other ML model or computer program, just mechanically executing its program without any conscious experience? What stops it from being, basically, a machine version of a philosophical zombie?
This is the hard problem of consciousness in a nutshell, and the Schmidhuber comment doesn't address it in any meaningful way.
Your link is quite technical.
edit: Out of curiosity, why would one consider him infamous?
Perhaps look up the case with Ian Goodfellow's work on GANs. Schmidhuber said the GAN paper doesn't give sufficient credit to his earlier work on predictability minimization and was very vocal about this also in conferences (interjecting etc.).
IIRC claims were made that LeCun etc. were unfairly credited for work pioneered by him. As in, he pretty much invented modern ML but wasn't credited at all.
If a simulation cannot produce consciousness, what is special about life that makes it unrepresentable with physics? Where is the boundary between a conscious system and a non-conscious one?
Conscious minds certainly seem bound by their physicality, you can figure out how feelings and experiences are represented molecularly and electrochemically and affect those feelings and experiences experimentally. So far we've not found anything that cannot be broken down into physics (not to say we won't), but if we assume there's nothing else _but_ physics – then I think we're left with:
- The physics consciousness relies upon is fundamentally uncomputable
- Consciousness _is_ computable and all the weird consequences that follow (like sand computers) are true
I lean towards the second option, but I struggle to understand it in any intuitive sense.
What are your thoughts on why a simulation cannot be conscious?
So, consciousness being computable may not necessarily imply that any given computer could do so. One potential way for this to be the case is if consciousness is tied to some form of hypercomputation (https://en.wikipedia.org/wiki/Hypercomputation), which can provide outputs that are not turing-computable. This would mean that a traditional computer couldn't simulate consciousness, but that another hypothetical machine might be able to.
Bostrom is assuming there's a thing called "computation" which is somehow a fully-specified solved problem. (It isn't).
Worse, he's also assuming that this "computation" can somehow simulate something called "consciousness" - which unfortunately for him, is nowhere near to having the formal rigorous definition that "computation" as we know it today would require.
And finally after all that, you get something which is subjectively indistinguishable from life itself.
Well - obviously. If you don't define any of your terms and assume your argument is correct because you don't show your working or get into specifics, you can argue whatever you like.
absolutely nothing
> The physics consciousness relies upon is fundamentally uncomputable
This would be "huge if true", but I don't think it is. At least I haven't seen any indication of why it would be.
> Consciousness _is_ computable and all the weird consequences that follow
I think this is true, and I think the weirdness is a result of misunderstandings on our part (we are notoriously bad when reasoning about systems that contain us in some way).
> What are your thoughts on why a simulation cannot be conscious?
I think this whole consciousness confusion is due to mixing levels of abstraction on our part. Replace consciousness with other higher-level cognitive or social functions, like "love" or "patriotism", and see how silly it sounds. Is "love" computable? Can "loyalty" be reduced to physics? Yes, but the concept makes sense on a whole different conceptual level, which would be silly to reduce to, say, quantum physics, and is much better to reason about at the right abstraction level.
For example, if I'm living in a simulation and I move my hand an inch to the left, it occupies an uncountably infinite number of intermediate positions during that motion. A Turing machine can't calculate or even represent all of those positions, since the set of Turing-computable numbers is only countably infinite.
Of course, what I've written above assumes that spacetime is continuous. Is it?
That can be tested by experiment:
https://physics.stackexchange.com/questions/9720/does-the-pl...
According to that link, spacetime shows no sign of discretization down to fourteen orders of magnitude below the Planck length. (The Planck length itself is so small that if an object that size were expanded so as to be a millimeter across, then a proton, at that scale, would be larger than the distance between the sun and Alpha Centauri.)
That suggests to me that while Turing machines can provide useful approximations to the behavior of physical systems, they can't represent or simulate them precisely.
Experimental evidence (the Fermi satellite data discussed in the link in the grandparent comment) seems to support standard physics.
Authors can write stories with believable worlds and compelling characters. Given infinite time, an author could detail how a character would react to every possible situation. But no matter how much they wrote down, their words would just be words. There isn't some breaking point where the author writes enough to create a pocket universe.
Why are turing machines different?
Just because the universe is governed by physics does not mean that physics are sufficient to construct a universe. I expect that "physicality" is a real property of some kind, which doesn't appear just because you described it accurately.
So much of this is guesswork, I may well be wrong—but it strikes me as the most logical conclusion.
In fact, it pretty much boils down to the same thing, as far as I can tell. You have a simulation? Someone set it up. He/she/it is the creator of this universe.
[Edit: I should say that a simulation winds up being the same as a religion, but I'm not sure that a religion necessarily winds up being the same as a simulation.]
Stating the cause/creator first and looking for evidence of it is not very reasonable.
On one hand it is a wonderful outlook where computation and imagination creates universes as real as our universe feels to us (or rather as real as each creatures individually experienced universe seems to them). On the other hand, it does concern me that our own and fellow creatures experiences are perceived by this school as formally attributable to controllable processes, distinguished in "reality" only by beholders individuated interpretations, be they someone, a convincing replication of someone, or anything really which any individual decides to treat as conscious or not.
Either: - It can, and so can rocks. - It can, but there's something else unknown that's necessary to generate consciousness. - It can't, consciousness doesn't exist.