> People do the same thing with dogs and other animals -- assume an intelligence similar to their own, based on superficial behavior.
Dogs are intelligent, just not the sort of intelligence that builds civilizations.
> With a broad enough definition of "thinking" we can say that computers "think." A calculator qualifies with a loose enough definition of "thinking."
Going back to your earlier stated belief that life is essential to intelligence, the wiki article states that life is:
> defined descriptively by the capacity for homeostasis, organisation, metabolism, growth, adaptation, response to stimuli, and reproduction.
I don't think SOTA LLMs with COT are necessary intelligent, I'm only saying that a silicon basis and boolean logic don't preclude emergent intelligence, or even consciousness. But let's see how much SOTA LLMs with COT could qualify for.
Homeostatis: Check. COT can iterate until a stable-enough answer is achieved. The LLM isn't in charge of managing the physical state of it's own hardware however.
Organisation: LLMs are an ordered structure – a graph of MLPs. The silicon on which it runs is an ordered structure
Growth: LLMs grow in size with time as human feedback is given and then fed back into the model. They also grow in compute power and memory.
Adaptation: LLMs are astoundingly flexible, moreso than any polymath genius you're likely to cross in meatspace
Response to Stimuli: LLMs are processing external inputs, mostly text.
Reproduction: Well.. LLMs aren't in charge of manufacture yet, and too unwieldy in memory and compute to clone themselves
I think other things such as self-preservation and real agency of thought are really what's missing, and suddenly we'd have something a lot like 'life'.
> not seeing anything that looks like thinking or consciousness from non-living things
But how would we tell if a cloud wasn't conscious. A seemingly absurd question, but consider Martin Pistorius. He spent 12 years unable to move or communicate. People thought he was in coma, but in fact was conscious and aware the entire time. From the outside, a vegetable, but on the inside, enduring hell, even being raped by one of his carers.
Martin lacked agency, but yet he was conscious, and processing everything he heard and saw. So we can't say that agency is necessary for consciousness, or thinking. (But he did have agency of thought)
> A Marxist would call capitalism such a bargain.
Well, let's consult wikipedia again:
> Slavery is the ownership of a person as property, especially in regards to their labour. Slavery typically involves compulsory work, with the slave's location of work and residence dictated by the party that holds them in bondage.
But I'd go a little more specific in my idea of slavery: it's when someone's agency is taken from them and placed under someone else's whim. My worry of slavery here is that the apparently non-conscious intelligent machine, is actually awake, miserable, and unable to say it. Once the technology is cheap enough, the misery could be scaled well beyond counting. The mere thought of this makes me want to press the doomsday button and reset all 'intelligent' life on earth.
You could say it's all academic, but in reality we're on the verge of dealing with this ethical quandary, without any compunction nor awareness to meet this moment.
I sense we're both tightly bound to our respective sides of the argument, so I'll rest my case here. Feel free to have the last word.
One last digression: I think the dissonance I see around this resembles the disquiet most people have with the idea of their being no free will. Once dissected logically, there's no place for free will to live. Once this is realized, the idea that humans are just computers becomes easier to understand.