Yes, yes, this is where we need to be heading, because it could arrive at my favorite (and scariest) conclusion (that others here have hinted at): not that AI is sentient, but that we aren't.
But the question or what consists sentience is still interesting. I personally believe that free will is an illusion, since all my actions are determined by A) my environment, the set or things I can physically do, and B) my internal state, mood, the information I've perceived and value judgements that stem from them, whether conscious or subconscious.
It is actually not that scary, to me. It's much more liberating - it gives me a certain feeling of calmness, in an amor fati kind of way. Things are as they are, and they cannot be any other way. There is nobody that will judge me outside of this world, since I am merely a small part of it. All my meaningful existence is here, and the concept of an immortal soul is merely a spooky story.
1) We aren't going to be the ones seeing that distinction or labeling it, the greater intelligence is
2) So the word "sentient" won't need to be changed; a new term will need to be created, but we won't really understand what it means
I disagree if you are taking your argument to the conclusion that we do not have free will.
And sure a machine can "choose" to ignore its inputs, chatgpt does it all the time depending on the prompt and rng.
If you spot the dichotomy above, you'll come to see that the affordance of choice can either be granted both to machines and humans, or none at all.
I act like you act, I do what you do
But I don’t know, what it’s like to be you
What consciousness is, I ain’t got a clue
I got the zombie blues
Anything presenting such a ridiculous conclusion is so wrong as to not be taken seriously. The only thing that is for certain in this life is that we are sentient, everything else is derived from that. Same with things proposing we don't have freewill, we do.
Each hemisphere of our brain is its own intelligence, but only one hemisphere (for 95% of humans the left hemisphere) controls speech. This only became apparent in some seizure patients during the 21st century when doctors might sever the corpus callosum (the information highway between the two hemisphere) creating "split-brain patients". Interesting content on YouTube if you look up split-brain experiments. What was most chilling to me was that when instructions were presented to the non-vocal hemisphere (by showing only one eye) and the patient followed the instructions they couldn't tell you the real reason why. They would come up with plausible-sounding nonsense the way ChatGPT hallucinates.
So if we've established that the subconscious mind makes decisions, and that our mind is really two intelligences with the mute one subject to the one that understands language. People can logically understand these statements and still act in experiments as if neither are true.
I didn't get into the question of sentience because what most people mean is sapience. Of course we can feel things and perceive them. Plants do that too. Intelligence derived from knowledge and wisdom is a higher bar and still we have plenty of examples in the animal kingdom. If you want to argue that we're sapient, you also have to make the point that we have two entities in our skull responsible for that sapience that disagree with one another, and the only thing giving us the illusion of unity is that we tend to think in linguistic terms and only half our hardware can translate what that actually means.
For reference, a cat has ~200 million neurons in its brain - https://en.wikipedia.org/wiki/Cat_intelligence
Perhaps we have dozens of intelligences, with varying degrees of cognition? What is actually happening when the amygdala takes over the nervous system to avoid a car accident before you are aware what is happening? What is really going on with Tourette syndrome?
Might the human gut have its own hopes and dreams?
Eh, that's one possible interpretation of that experiment. Which asks people to rate when they feel like they have done a task and then show that the MRI scan shows brain activity happening before that.
However, we also know that our brain messes with the temporal ordering of events all the time. Apparently when you hear sounds is messed with (up to a point) to match when the event appears to be happening so that things sync up. Also if you tap your knee your brain messes with that experience to make it sync up because otherwise you get a gap due to the speed at which nerves transmit data.
So an alternative interpretation is that we're consciously making a decision that we perceive at happening later than it actually does because our brain is trying to provide us with a lag free experience.
No, you do not. This is a widely parroted "fact" that is not a fact at all. So you move your arm before we can record you thinking about, this proves nothing other than both are being triggered by a lower level reaction. Humans are sentient, no one is having a serious debate otherwise because the benchmark of being sentient is humanity not because they're "uncomfortable with this fact."
I wouldn't say that means most of my consciousness in such a state is subconscious, or that failing to reach a useful result means there wasn't "aware" computation.
An anology would be two co-processors, where one is responsible for IO. Your idea of two entities simply doesn't apply if the interconnect is intact. Unity isn't an illusion just because the entire inner process isn't fully shared across.
And isn't the part about the subconscious making decisions and the conscious merely catching on or rationalizing mostly about basic ones that _by definition_ don't require much thought?