I think this writer is missing the core reason we dismiss sentience claims of these kinds of known mechanical processes. Inference as carried about by a LLM can, in principle, be simulated by mechanical switches, humans passing index cards to each other, or any other means of simulating individual action potential switches in a known sequence. It might take centuries for a single inference to take place, but it can be done. You can take the same inputs and produce the same output as LaMDA. Yet we seemingly know for certain humans passing index cards to each other don't achieve any sort of group super-sentience. The sentience boundary ends at the borders of each individual brain.

What we don't understand is why this is, but we know it is. The fact that I can communicate and give instructions to other people and we can collectively compute more than I can individually compute doesn't make any of us aware of what the other is experiencing subjectively. This isn't a novel observation. It's just the Chinese Room argument, but applied to sentience rather than understanding. Sentience is in some way, for some reason, tied to a specific bounded substrate. It doesn't cross air gaps, and is at least in that way, not a result purely of the nature of how information processing is happening. It has to involve more than that, even if we don't know and possibly can't know what.

So how does that analogize to automated computations happening with electricity? Is it just the air gap? If we implanted radio chips in each other's brains and were able to achieve telepathic communication in this way, would we become aware of each other's subjective experiences or would we just have a faster comm link than audio waves? Is it related to some unknown physical quality of neurological material? If we were able to graft direct neuronal links between two people, would their experience change to become the experience of one person? Studies of split brain patients who have had their corpus callosum severed actually suggest this might be the case, but we have never tried it in the other direction (and I can't think of an ethical way to carry about such an experiment).

But again, how does this analogize to an electronic computer? Does a LLM possibly achieve sentience if all of its nodes are physically wired together, but not if any of them communicate via radio? Or is the problem of the air gap in humans a matter of signal delay and not the physical material passing the signal? Perhaps when we implant radio chips and achieve telepathic communication, we really will become group hive minds without individual subjective identities. We won't know until we try.

All we know for certain right now is that sentience and the perception of an individual self only happens in bounded physically connected computers composed of biological material. Given split brain studies, physically connected probably for sure matters, but we can't know it is that and not signal delay without being able to graft a neuron-speed comm relay that is not biological into biological brains. Nonetheless, I think a lot of people have a strong intuition that, for some reason or other, the physical substrate itself matters. That is to say, those subcellular processes not being modeled by a LLM are somehow involved in producing sentience.