What's your objection to "conscious" machines? (Putting it in quotes because that's a loaded, unscientific term and everybody has their own definition)
Dr. John Searle's objection is that software is not a mind, and goes on to explain why with his Chinese Room thought experiment.[1]
That's an argument for why we _shouldn't_ build them. But not an argument for why we _can't_ build them. Do you see the difference?
> So how are we supposed to make a machine conscious without knowing how we ourselves are conscious?
Evolution didn't 'know' about conscious either.
The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.)
For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room
Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox.
This is a straw man fallacy, because I was asked "What's your objection to "conscious" machines?", and my objection is that it is cruel.
> Evolution didn't 'know' about conscious either. The pioneers of the Industrial Revolution had only the faintest ideas about thermodynamics, yet managed to build working steam engines. (And we improved them over time, as we learned more.)
This is also a fallacious argument. Just because we don't know how evolution created consciousness doesn't mean that we can use our ignorance of how consciousness evolved to create machine consciousness. If you're only using this as evidence that consciousness is possible, the fallacy is begging the question, petitio principii.
Also, I believe there is here equivocation and conflation of artificial intelligence with natural intelligence. If we have the time, we can spend 600M years to evolve consciousness directly, but then it would not be artificial, it would not be machine intelligence, it would be natural or biological intelligence, and it would be a massive waste of time because it is already here and we are it.
> For the 'Chinese Room' see eg https://www.scottaaronson.com/democritus/lec4.html or just have a look at good old Wikipedia: https://en.wikipedia.org/wiki/Chinese_room
Do you really expect me to argue against your citations when you're too lazy even to summarize any particular counter-argument? It has been said that most of the work done in AI has been in trying to prove John Searle wrong, which suggests his argument is significant. A Turing Test does not prove consciousness nor even intelligence. All it proves is whether a human can be fooled. Arguments suggesting emergent properties between a human and a rule book are patently ridiculous, absurd on their face, because learning a book could not possibly ever make a book conscious. Where the Chinese Room thought experiment breaks down is there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a computer where these types of counter-arguments would ever come true: no matter how much machine learning you submit your program to, there is no mind to begin with that could ever become aware.
> Though I'm not sure why you brought up consciousness in the first place. It's irrelevant for the Fermi Paradox.
In fact, I did not, the OP of the tread asserted: "Seems pretty clear that we're on a path to machine intelligence," and I responded to that specifically and argued against the possibility.
OK, you want to argue both against the _should_ and the _could_, I guess?
About the Chinese Room: sure I can replay the standard arguments, if you want me to.
But if you already reject the validity of the Turing test, there's not much to discuss on that front, I guess.
How do you know that you aren't the only conscious being in existence? How do you know whether other commenters on HN are conscious?
> It has been said that most of the work done in AI has been in trying to prove John Searle wrong, which suggests his argument is significant.
Who said that? Lots of other things have been said, too. That doesn't suggest anything.
> Where the Chinese Room thought experiment breaks down is there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a computer where these types of counter-arguments would ever come true: no matter how much machine learning you submit your program to, there is no mind to begin with that could ever become aware.
Aren't you assuming what you want to prove here?
Btw, would your argument still apply in this form?:
> [...] there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a [brain] where these types of counter-arguments would ever come true: no matter how much [of anything] you submit [a brain] to, there is no mind to begin with that could ever become aware.
OK...? Strong AI is not possible for reasons given. Strong AI, if possible, is slavery, thus unethical.
> About the Chinese Room: sure I can replay the standard arguments, if you want me to. But if you already reject the validity of the Turing test, there's not much to discuss on that front, I guess.
Straw man, I have not rejected the validity of the Turing test, merely making the observation that a program passing a Turing test is merely fooling a human into uncertainty whether they are interacting with a human or a program. The Turing test itself is not proof of consciousness, it is only proof of a human being fooled.
> How do you know that you aren't the only conscious being in existence? How do you know whether other commenters on HN are conscious?
Solipsism and problem of other minds, and as such, a trap. Knock yourself out: https://plato.stanford.edu/entries/other-minds/
> Aren't you assuming what you want to prove here?
Nope.
> Btw, would your argument still apply in this form?: > [...] there is a human involved, and most counter-arguments get tripped up here forgetting that it is metaphor and there is no human inside a [brain] where these types of counter-arguments would ever come true: no matter how much [of anything] you submit [a brain] to, there is no mind to begin with that could ever become aware.
No, this is building a straw man and sounds a lot like homunculus theory.
Either English is not your first language, or you are just playing stupid games to avoid answering. or maybe both.
You were clearly not asked if you though it was immoral, you were clearly asked why you think it's impossible.
I honestly don't think that anyone would make that specific argument. Do you think that they are? Who's suggesting that we "use ignorance to create consciousness"? It seems an odd diversion to take.
> If you're only using this as ( consciousness has evolved) evidence that consciousness is possible,
Is it evidence that consciousness is possible? I think it is.
Is it evidence that consciousness is possible, from only ordinary matter and energy and no divine spark? I think it is.
Is that an argument that a constructed machine consciousness is also not ruled out? I think it is.
> the fallacy is begging the question, petitio principii.
I don't know what you're getting at here, or why.
I always found that experiment to be extremely unconvincing, to put it mildly. Using that framework I can prove that you are not conscious.
I construct a "english room". You are in the back without any contact to the outside world. A Chinese speaker in the front can pass messages to you from the outside world, in English by laborious copying. Clearly you aren't conscious because the Chinese speaker can't understand the English.
Or, alternatively, no one would dispute a single neuron is not conscious and can't understand english. I pass signals into your neurons and through some electrochemical magic you reply in english, yet the neurons involved don't understand english therefor you aren't conscious.
All Searle has done is picked out one component of a composite system, concluded that in isolation it has no clue whats going on, and from that concluded the system as a whole doesn't.
We have a hard time clearly defining consciousness, and it's not clear if some AGI would be conscious (maybe it's just an emergent phenomena)-- it's also not clear if there would be any particular advantage in it being so. But in any case, I don't think the chinese room is a useful argument for anything.
Our own brains are an excellent proof of concept that machines can be made to think. We're all machines in a way, after all - unless you believe there is a special supernatural exception for human minds.
A machine is an artifact. It is a human-built assembly of materials that have no natural inclination to work in unison. Silicon and copper, plastic and glass have no natural propensity to compute. We make artifacts, for example computers, out of these materials to serve our purposes. But they are not the materials’ natural purposes.
In the classical hylemorphic (Aristotelian) perspective, an artifact is an artificial composite of substances and thus it has an accidental, not substantial, form. Machines work as they do “accidentally”; that is, their function as machines is not inherent to the matter that comprises them. Rather their function is imposed on the disparate parts by human intelligence.
Unlike a machine, the brain is an organ, a functional part of a living organism. It (along with the body) has a substantial form; its activity is natural to it. One might say that the brain's activity is intrinsic to it, not imposed on it extrinsically, as is the activity of a machine.[1]
[1] https://mindmatters.ai/2018/10/yes-your-brain-is-a-machine-i...
So we looked and saw that life is just molecular nanotech, but what we didn't see was any tags attached to carbon chains that would say, "this molecule's natural function is to think", or "that molecule's natural function is to be a part of an organ that pumps blood", etc. Because there is no such thing like a "natural function" in the sense used by your quote, the distinction between "natural" and "artificial" is itself artificial[0], and that whole quote represents a piece of magical thinking philosophy you'd think humanity abandoned at least a century ago.
----
You could just let the discourse end there - it's a strongly held belief and that's the end of the discussion. What confuses me, and maybe you can clear this up in a reply, that you still want to have a discussion about facts and reasoning. For that to happen, you would need to show that "nature" does indeed have the privileged and unreplicable position you assert it does. Just using the term "nature" is rhetorically evocative but not rationally meaningful in this context. To start with, you'd also have to definitionally delineate nature from artifice, because we humans have really been blurring those lines for a long time now and no cosmic force has stopped us yet.
Your implication that "natural" processes can never be functionally replicated or continued upon by volitional development is not supported by evidence. There is, however, a lot of evidence for the other side of this debate. We are heavily altering "nature" to suit us, and wherever we looked so far we find machinery that can be rationally understood, manipulated, replicated, remixed, and used whole or in part in our own designs. If there is a supernatural component that somehow privileges "natural" life, we have not found it yet.
and the rest of your paragraph is about artificial vs natural, which isn't even relevant, even if it could be clearly defined. Asserting "Proof by assertion is fallacious argument" is a fallacious argument.
I don't think that you understand parent comment's very valid point.
If the question is "Can genuine (human-level) consciousness be constructed out of nothing but atoms and energy?" then the only possible answers are:
1 yes, and at human scale, since a human being is a worked example of such, made from nothing but atoms and energy.
2 no, a human being is not just made of matter and energy, there is some "extra" special part to a human, call it soul, spirit, divine spark, etc without which there is not "genuine consciousness". (a)
And the whole rational project of science, going back 100s years, dismisses option 2 as sheer useless mystical mumbo-jumbo with no explanatory power at all. Therefor, option 1 is valid.
a)