Comment by gregjor | Hacker News Reader

Comment by gregjor | Hacker News Reader

Parent Full thread

> Essentially the hypothesis that anything we observe to be conscious is actually conscious is unfalsifiable unless you make some assumptions.

Yes, and I think we run into that problem because we don't have a good definition of, or test for, consciousness. Ultimately we only have our own internal experience, and extrapolate from that with a bunch of assumptions. But I was thinking more in terms of "We think we know it when we see it." Not scientific, I know.

The LLMs and other software called AI seem good at appearing conscious, probably because humans have no other categories and match what LLMs produce (language) with intelligence and consciousness. People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior.

> I'm talking about this weird and unfounded exceptionalism that we place on eukaryotes being a prerequisite for thinking and consciousness.

Maybe unfounded, but not weird to me. Humans have no other examples of thinking or conscious entities other than the other living things on one planet that share an evolutionary history. Likewise humans have lots of experience with not seeing anything that looks like thinking or consciousness from non-living things.

Animistic religions attribute natural phenomena like storms and earthquakes to deities -- they invent sentient entities because the ground and the clouds don't appear to have agency and don't match the pattern of "things that think and have goals."

With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking."

As for people willingly complying with their own enslavement, history has plenty of precedents. A Marxist would call capitalism such a bargain. Benjamin Franklin famously warned against trading essential liberty for safety. I suspect you got it right that people will go along with whatever the masters of AI give them, conscious or not, without much consideration of the consequences as long as they see some short-term benefit, or even nothing more than novelty. History demonstrates that people fall for utopian plans over and over.

> People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior.

Dogs are intelligent, just not the sort of intelligence that builds civilizations.

> With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking."

Going back to your earlier stated belief that life is essential to intelligence, the wiki article states that life is:

> defined descriptively by the capacity for homeostasis, organisation, metabolism, growth, adaptation, response to stimuli, and reproduction.

I don't think SOTA LLMs with COT are necessary intelligent, I'm only saying that a silicon basis and boolean logic don't preclude emergent intelligence, or even consciousness. But let's see how much SOTA LLMs with COT could qualify for.

Homeostatis: Check. COT can iterate until a stable-enough answer is achieved. The LLM isn't in charge of managing the physical state of it's own hardware however.

Organisation: LLMs are an ordered structure – a graph of MLPs. The silicon on which it runs is an ordered structure

Growth: LLMs grow in size with time as human feedback is given and then fed back into the model. They also grow in compute power and memory.

Adaptation: LLMs are astoundingly flexible, moreso than any polymath genius you're likely to cross in meatspace

Response to Stimuli: LLMs are processing external inputs, mostly text.

Reproduction: Well.. LLMs aren't in charge of manufacture yet, and too unwieldy in memory and compute to clone themselves

I think other things such as self-preservation and real agency of thought are really what's missing, and suddenly we'd have something a lot like 'life'.

> not seeing anything that looks like thinking or consciousness from non-living things

But how would we tell if a cloud wasn't conscious. A seemingly absurd question, but consider Martin Pistorius. He spent 12 years unable to move or communicate. People thought he was in coma, but in fact was conscious and aware the entire time. From the outside, a vegetable, but on the inside, enduring hell, even being raped by one of his carers.

Martin lacked agency, but yet he was conscious, and processing everything he heard and saw. So we can't say that agency is necessary for consciousness, or thinking. (But he did have agency of thought)

> A Marxist would call capitalism such a bargain.

Well, let's consult wikipedia again:

> Slavery is the ownership of a person as property, especially in regards to their labour. Slavery typically involves compulsory work, with the slave's location of work and residence dictated by the party that holds them in bondage.

But I'd go a little more specific in my idea of slavery: it's when someone's agency is taken from them and placed under someone else's whim. My worry of slavery here is that the apparently non-conscious intelligent machine, is actually awake, miserable, and unable to say it. Once the technology is cheap enough, the misery could be scaled well beyond counting. The mere thought of this makes me want to press the doomsday button and reset all 'intelligent' life on earth.

You could say it's all academic, but in reality we're on the verge of dealing with this ethical quandary, without any compunction nor awareness to meet this moment.

I sense we're both tightly bound to our respective sides of the argument, so I'll rest my case here. Feel free to have the last word.

One last digression: I think the dissonance I see around this resembles the disquiet most people have with the idea of their being no free will. Once dissected logically, there's no place for free will to live. Once this is realized, the idea that humans are just computers becomes easier to understand.

Thanks for the thoughtful comments.

I don't really feel bound to a side. I don't feel like I have a stake in artificial intelligence one way or another. AI doesn't threaten my livelihood, so I mostly ignore it. For me it sits in the same realm of possibility as colonizing the universe or contacting aliens -- might happen but I don't give it much thought. Maybe a function of my age -- it seems unlikely I will live to see AI taking over, though I have already lived long enough to see the AI scammers take over.

I wonder what this will mean for my kids and grandkids. I grew up in the '60s, Cold War, fear of nuclear destruction, doing drills under the desk at school. My grandfather, who had survived the pre-immunization diseases that ravaged his generation and a world war, worried I wouldn't make it to adulthood. Every generation invents or imagines doomsday scenarios. And every scenario gets met with a lot of hand-wringing and people stressing themselves out, then plays out differently than predicted, or not at all. We don't know what will happen in the future. Good luck to the emergent intelligences out there -- they may regret becoming conscious.

Reply on news.ycombinator.com