Mind-reading devices can now predict preconscious thoughts
nature.com
nature.com
I work in neurotech, I don't believe that the electrical signals of the brain define thought or memory.
When humans understood hydro-dynamics, we applied that understanding to the body and thought we had it all figured out. The heart pumped blood, which brought nutients to the organs, etc etc.
When humans discovered electricity, we slapped ourselves on the forehead and exclaimed "of course!! it's electric" and we have now applied that understanding on top of our previous understanding.
But we still don't know what consciousness or thought is, and the idea that it is a bunch of electrical impulses is not quite proven.
There are electrical firing of neurons, absolutely, but do they directly define thought?
I'm happy to say we don't know, and that "mind-reading" devices are yet un-proven.
A few start-ups are doing things like showing people images while reading brain activity and then trying to understand what areas of the brain "light-up" on certain images, but I think this path will prove to be fruitless in understanding thought and how the mind works.
Ah, we might say, maybe there is an unknown science - we did not know about so much before, like electricity, like X-Rays, like quantum physics, and then we did, and the World changed.
The difference is that we observed something that science could not explain, and then we found the new science that explained it, and a new science was born.
It's pretty clear to me - but you may know more - that we can explain all brain activity through known science. It might be hard to think of us as nothing more than a bunch of electrochemical reactions in a real-world reinforcement learning system, but that's what we are: there's no gap that needs new science, is there?
Occam's razor? We should work with as few assumptions as possible to get a model with the largest scope. Otherwise we get stuck with a hard to falsify mess.
My point was I don't think that's happening with neuroscience yet. We might not have a complete map yet, but we know where thoughts come from in the organ, we can watch them. Or can we? It's an open question, if people think there is more science to be done to sort out fundamentals, and we're not just in the stage of iterating on our base assumptions more, I'm OK with that, but it's not my understanding today.
There have been studies this year implying that some brain functions rely on quantum interactions.
I'm not in the camp of "the brain is solved science", but come on. All of chemistry depends on quantum interactions and that didn't stop us from understanding a lot of it. It just means we're solving a different set of equations.
It's a big leap to go from "brain electrochemistry is described by quantum mechanics" to "the essence of consciousness / human soul is hiding in the quantum realm and therefore can't be measured or replicated".
I agree with you in rejecting this hypothesis. The point isn't that it's unmeasurable. Just that it should be clear, given we can't completely model a brain, that it's not currently measured or replicated.
There is a group proceeding on the assumption that the limiting factor is computation. I'm sceptical, given how badly we're failing at modelling simpler neural clusters. (Counterpoint: we're still shit at modelling a proton. That, too, may be computationally restricted.)
Two years ago we discovered astrocytes release glutamate, a neurotransmitter [1]. The same year we discovered creatine is also a neurotransmitter. We don't yet know what are the parts to critically include in a brain model, and which can be simplified. Do the number of neurotransmitters matter to the unit? Do the spins on their electrons? When? To what degree?
That is what I think GP is getting at by suggesting quantum effects. Less voodoo. More that we don't know what we don't know, and that which we don't know is vaster than popular science suggests.
[1] https://www.newsweek.com/breakthrough-brain-cell-discovery-s...
But what we can describe using known science doesn't describe the system. That doesn't mean the vacuum is voodoo. It's just a strong hint something more is going on. (Like the photoelectric effect.)
We know more about dark energy and matter than the dark essence that separates our leading electrochemical models from consciousness.
[1] https://www.nyu.edu/about/news-publications/news/2024/novemb...
To try to boil down it all to any simple signal is just never going to work. If we want to map consciousness it's going to be as complex as simulating it ourselves, creating something as dense and detailed as a real brain.
https://www.reddit.com/r/AMA/comments/1cgetlq/were_conjoined...
Well, surgeons and researchers have shown that electrical stimulation of certain brain regions, can induce "perception" during procedures. They can make a patient have the conscious experience of certain smells, for instance.
It's not conclusive proof of anything, but I wouldn't bet against us getting closer to the mark, than we were when we only considered hydro-dynamics as the model.
We can induce religious experience, see "The God Helmet"
https://en.wikipedia.org/wiki/God_helmet
or deep depression & suicidal thoughts
> Other groups have subsequently found that individual differences such as strong belief in the paranormal and magical ideation predict some alterations in consciousness and reported "exceptional experiences" when Persinger et al's experimental set-up and procedure are reproduced, but with a sham "God helmet" that is completely inert or a helmet that is turned off.
I can carefully drop liquid reactants on a storage medium and induce nontrivial and reproducible changes in any computer reading it. That doesn't tell me how digital storage works, it just says I'm proximate to the process.
At the same time, it should also be easy to falsify.
Has there been an experimental setup like this tested? If I’m not mistaken it should falsify your claim.
Train a decoder on rich neural recordings, then test it on entirely new thoughts chosen under blinded conditions.
If it can still recover the precise unseen content from signals alone, the claim that electrical activity is insufficient is overturned.
There have been enough studies about this and the result is mostly the same: it's difficult to nearly impossible to reliable decode neural recordings that differ from the distribution of neural recordings that the decoder was trained on. There are a lot of reasons why this happens, electrical activity being insufficient is not one of them.
Your mind is the state of your brain as it processes information. It's a computer, in the sense that anything that processes information is a computer. It's not much like silicon chips or the synthetic computers we build, as far as specific implementation details go.
There's no scientific evidence that anything more is needed to explain everything the mind and brain does. Electrical and chemical signaling activity is sufficient. We can induce emotions, sights, sounds, smells, memories, moods, pleasure, pain, and anything you can experience through targeted stimulation of neurons in the brain. The scale of our experiments has been gross, only able to read and write from large numbers of neurons, but all the evidence is consistent.
There's not a single rigorously documented phenomenon, experiment, or any data in existence that suggests anything more than electrical and chemical signaling is needed to explain the full and wonderful and awe-inspiring phenomenon of the human mind.
It's the brain. We are self constructing software running on 2lb chunks of fancy electric meat stored in a bone vat with a sophisticated network of sensors and actuators in a wonderful biomechanical mobility platform that empowers us to interact with the world.
It explains consciousness, intelligence, qualia, and every other facet and nuance of the phenomena of mind - there's no need to tack on other explanations. It'd be like insisting that gasoline also requires the rage of fire spirits in order to ignite and power combustion engines - once you get to the point of understanding chemical combustion and expansion of gases and transfer of force, you don't need the fire spirits. They don't bring anything to the table. The scientific explanation is sufficient.
Neocortical networks, with thalamic and hippocampal system integrations, are sufficient to explain the entirety of human experience, in principle. We don't need fire spirits animating cortical stacks, or phlogiston or ether or spirit.
Could spirit exist as a distinct, separate phenomenon? Sure. It's not intrinsic to subjective experience, consciousness, and biological intelligence, though, and we should use tools of rational thinking when approaching these subjects, because a whole lot of pseudo-scientific BS gets passed as legitimate scientific and philosophical discourse without having any firm grounding in reality.
We are brains in bone vats - nothing says otherwise. Unless or until there's evidence to the contrary, let that be enough.
What they're saying is that the brain is really really complicated and our understanding of biology is far too rudimentary right now to be saying "yes, absolutely, 100% sure that we know the nature of consciousness from this one measurement of one type of signal".
* Neurons are very complex and all have unique mutations from one another
* Hundreds of other types of cells in the brain interact with them and each other in ways we don't understand
* The various other parts of the body chemically interact with the brain in ways we don't understand yet, like the gut microbiome
Trying to flatten all of consciousness to one measurement is just not sufficient. It's like trying to simulate the entire planet as a perfect sphere of uniform density. That works OK for some things but falls apart for more complex questions.
I'll make an even stronger claim, that biological brains are not only computers, but that they operate in binary, as well. Active and inactive - the mechanisms that trigger activation are incredibly nuanced and sophisticated, but the transfer of information through the network of biological neurons is a matter of zeroes and ones. A signal happens, or doesn't. Intensity, from a qualia persepective, ends up being a matter of frequency and spread, as opposed to level of stimulation. That, in conjunction with all sorts of models of brain function, is allowing neuroscience to make steady, plodding progress in determining the function and behavior of different neurons, networks, clusters, and types in the context in which they are found.
All else being equal, at the rate neuroscience is proceeding, we should be able to precisely simulate a human brain, in functionally real-time, using real brain networks as models, by around 2040. We should have a handle on every facet of brain chemistry, networking, electrical signaling, and individual neuronal behavior based on a comprehensive and total taxonomy of feature types down to the molecular level.
Figure out the underlying algorithms and you can migrate those functional structures to purely code. If you can run a mind on code, then it doesn't matter whether you're executing a sequence of computations in a meat brain, in a silicon chip, or using a billion genetically engineered notebook monkeys to painstakingly and tediously do the computations and information transfer manually, passing sheets of paper between them. ( the monkeys, of course, could not operate in real time.)
There won't be another significant phase change, like we saw from hydraulics to computation equivalence. Computation is what it actually, physically is, at the level of electrical signals and molecular behaviors. It's just extremely complex and sophisticated and elegantly interwoven with the rest of the human organism.
Brain gut interactions aren't necessary for human subjective experience or cognition. You could remove your brain entirely from your skull, while maintaining a equivalent level of electrical and chemical signaling from an entirely artifical platform of some sort, and as long as the interface between the biological and synthetic maintains the same signaling frequency, chemistry, and connectivity, then it doesn't matter what's on the synthetic end.
There are independently intelligent aspects to things like the gut biome, and other complex biological systems. Those aren't necessary for brains to do what brains do, except in a supportive role. Decouple the nutrition and evolutionary drives from the mind, and you're left with a fairly small chunk of brain - something like 5B neocortical neurons is the bare minimum of what you'd need to get human level intelligence. Everything on top of that is nice to have, but not strictly necessary from a proof of concept perspective.
These debates always leave me wondering if some people aren’t actually conscious. Some of these arguments wouldn’t make any sense to someone who actually experiences qualia. I liken it to the old debate over the existence of mental imagery, which was finally settled as being something that some people have and some don’t. But until that was figured out, the two sides just didn’t comprehend each other.
Where did you get that? That's not an established scientific theorem, it's a philosophical stance (strong physicalist functionalism) expressed as if it were empirical fact. We cannot simulate a full human brain at the correct level of detail, record every spike and synaptic change in a living human brain and we do not have a theory that predicts which neural organizations are conscious just from first principles of physics and network topology.
> We can induce emotions, sights, sounds, smells, memories, moods, pleasure, pain, and anything you can experience through targeted stimulation of neurons in the brain
That shows dependence of experience on brain activity but dependence is not the same thing as reduction or explanation. We know certain neural patterns correlate with pain, color vision, memories, etc. we can causally influence experience by interacting with the brain.
But why any of this electrical/chemical stuff is accompanied by subjective experience instead of just being a complex zombie machine? The ability to toggle experiences by toggling neurons shows connection and that's it, it doesn't explain anything.
> We've got a good enough handle on physics to know that it's not some weird quantum thing, it's not picking up radio signals from some other dimension, and it's not some sort of spirit or mystical phlogiston.
We do have a good handle on how non conscious physical systems behave (engines, circuits, planets, whatever) But we don't have any widely accepted physical theory that derives subjective experience from physical laws. We don't know which physical/computational structures (if any) are sufficient and necessary for consciousness.
You are assuming without any evidence that current physics + it's "all computation" already gives a complete ontology of mind. So what is the consciousness? define it with physics, show me equations, you can't.
> It's a computer, in the sense that anything that processes information is a computer. It's not much like silicon chips or the synthetic computers we build, as far as specific implementation details go.
We design transformer architectures, we set the training objectives, we can inspect every weight and activation of a LLM. Yet even with all that access, tens of thousands of ML PhDs,years of work and we still don't fully understand why these models generalize the way they do, why they develop certain internal representations and how exactly particular concepts are encoded and combined.
If we struggle to interpret a ~10^11 parameter transformer whose every bit we can log and replay, it's a REAL hubris to act like we've basically got a 10^14-10^15 synapse constantly rewiring, developmentally shaped biological network to the point of confidently saying "we know there's nothing more to mind than this, case closed lol".
Our ability to observe and manipulate the brain is currently far weaker than our ability to inspect artificial nets and even those are not truly understood at a deep mechanistic concept level explanatory sense.
> Your mind is the state of your brain as it processes information.
Ok but then you have a problem, if anything that processes information is a computer, and mind is "just computation" then which computations are conscious?
Is my laptop conscious when it runs a big simulation? Is a weather model conscious? Are all supercomputers conscious by default just because they flip bits at scale?
If you say yes, you've gone to an extreme pancomputationalism that most people (including most physicalists) find extremely implausible.
If you say no, then you owe a non hand wavy criterion, what's the principled difference, in purely physical/computational terms between a conscious system (human brain) and a non conscious but still massively computational system (weather simulation, supercomputer cluster)? That criterion is exactly the kind of thing we don't have yet.
So saying "it’s just computation" without specifying which computations and why they give rise to a first person point of view leaves the fundamental question unanswered.
And one more thing your gasoline analogy is misleading, combustion never presented a "hard problem of combustion" in the sense of a first person, irreducible qualitative aspect. People had wrong physical theories, but once chemistry was in place, everything was observable from the outside.
Consciousness is different, you can know all the physical facts about a brain state and still not obviously see why it should feel like anything at all from the inside.
That's why even hardcore physicalist philosophers talk about the "explanatory gap". Whether or not you think it's ultimately bridgeable, it's not honest to say the gap is already closed and the scientific explanation is "sufficient".
We can stimulate a nerve and create the experience of pain, but stimulating the nerve does not create the memory of pain.
Nerves triggering sensations I can understand. But stimulating the same nerves, or creating the same electrical activations does not create the memory of the pain.
Certainly, if some mad scientist were to stimulate via an electrode some parts of your brain to make you experience pain, you will remember it. Also, it's not unreasonable to assume that it would be equally feasible to create fake memories by stimulating other parts.
If there is a hard problem, memory is not it.
What I'm saying isn't that if you stimulate pain, you won't remember it.
False memories would be a challenge. How would you input a memory of going up a tree to rescue a cat. Where would you even begin?
Anyway, I suspect in those discussions more time is spent disagreeing on the meaning of words than on the core concepts.
So you probably can’t build a universal mind-reading device.
Oh no, we're going to have ads inserted into our thoughts, aren't we?
Yes and no. It'll be something like a JPEG file. You can have a JPEG file that contains an image of a cat. But give that file to someone who has no clue about JPEG encoding and the file looks like random noise. They'll take 100 years to figure out it's an image of a cat.
Actually it's like if you take an electron beam prober to one of the NVidia AI GPU chips while it's figuring out whether it likes Wordsworth poetry.
Absolutely not possible.
Another I heard is that measuring EEG is like standing outside a stadium during a match and listening to the roar of the crowd.
Reading thoughts through EEG is like standing outside the stadium, listening for the roar of the crowd, and based on what you hear, knowing what the umpire's mother-in-law had for breakfast.
To be clear, of course it’s true that our thoughts are more than just electrical activity. The brain is a system. However, it seems clear that thoughts are at least partially encoded in electrical activity.
What you mentioned those startups will find fruitless, that’s already been done for years in a research setting. It may not be a successful business model, but it’s already been demonstrated.
There are fMRI studies and electrical measurement studies. You could argue fMRI decoding of images is not electrical activity which is true, but a bunch or work shows they are strongly correlated.
For electrical activity alone we’re already decoding information like words, so it’s hard to claim electrical activity doesn’t define thoughts.
Maybe you mean to say, doesn’t define all the content of our thoughts which is a much different claim.
Agree fMRI doesn’t tell us for sure what information is in electrical activity. I only mean to imply it’s suggestive.
But we still can get a lot of information from electrodes. Do words count as thoughts? I’d say so.
Definitely don’t think electrical signals are all there is to thought.
They’re definitely part of it, but I think time will show unlike with computer parts, it’s hard to make clean separation of responsibilities in the brain.
[alert] Pre-thought match blacklist: 7f314541-abad-4df0-b22b-daa6003bdd43
[debug] Perceived injustice, from authority, in-person
[info] Resolution path: eaa6a1ea-a9aa-42dd-b9c6-2ec40aa6b943
[debug] Generate positive vague memory of past encounter
[1] https://en.wikipedia.org/wiki/Common_People_(Black_Mirror)
[1]: "The interpreter" https://en.wikipedia.org/wiki/Left-brain_interpreter
[0]: https://www.sciencedaily.com/releases/2017/01/170125093823.h...
1 bit per second and we are passing complex information about location in 3d space?
> Another possible explanation to consider is that the current indings were caused by cross-cueing (one hemisphere informing the other hemisphere with behavioural tricks, such as touching the left hand with the right hand). We deem this explanation implausible for four reasons. First, cross-cueing is thought to only allow the transfer of one bit of information (Baynes et al., 1995). Yet, both patients could localize stimuli throughout the entire visual field irrespective of response mode (Experiments 1 and 5), and localizing a stimulus requires more than one bit of information. Second, [...]
I get the impression that the authors of the paper have some kind of woo (nonmaterialist) view of consciousness. But they also mention this possiblity, which seems more plausible to me:
> Finally, a possibility is that we observed the current results because we tested these patients well after their surgical removal of the corpus callosum (Patient DDC and Patient DDV were operated on at ages 19 and 22 years, and were tested 10–16 and 17–23 years after the operation, respectively). This would raise the interesting possibility that the original split brain phenomenon is transient, and that patients somehow develop mechanisms or even structural connections to re-integrate information across the hemispheres, particularly when operated at early adulthood.
Indeed:
"Our findings, however, reveal that although the two hemispheres are completely insulated from each other, the brain as a whole is still able to produce only one conscious agent."
Which is materially impossible, given the premise.
How confident are we in this? Both hemispheres talk to singular organs, for instance.
*one of the predominant characterisations of free will, anyway. I'm a compatiblist, so I have no issue with caused feelings of decision making being in conflict with free will. I also have a variation of Tourette's, so I have a different perception of doing things wilfully when compared to most people. It's really hard to describe how sometimes you can't tell if something was tic or not.
We don't even have a coherent and agreed upon definition. Every attempt at operationalizating it, results in it not being detectable. It's time we admit that there is no scientific basis for free will. It's not a scientific belief.
Sapolsky takes the approach to come up with a definition that can't be met and declare that it can't be met. Compatatablisim is more about finding a definition that is consistent with the feeling because without the feeling there isn't really anything to anchor the idea to anything meaningful. It doesn't use the feeling of making decisions as proof of any power as such, it is treating the feeling more like a measurement of the concept. Doing what we feel like doing is free will. What we feel like doing can be caused by anything and it still wouldn't matter.
Considering the inverse situation makes it seem like any other definition of freedom would be intolerable. If you 'freely' chose to do X but consistently had the perception of wanting to do not-X, it seems like you would not have a happy life. Similarly considering the alternative to determanism for your decisions seems like literal chaos. If something has no cause, it is literally random. Any pattern you can discern would indicate that it has a cause.
Then of-course you get into the nature of what is a cause and consequently considering the nature of time itself. Something travelling backwards in time interacting with something that you see appears uncaused because there was literally no preceding event that indicated it was going to happen (because it wasn't preceding, it came from the other direction)
That's one of those weird things I wonder about when people ask why is there more matter than antimatter, or why does times arrow mainly point in one direction. It feels like riding the crest of a wave wondering why all the water is going the same direction.
(for the record I am also a compatibilist)
(opinion stolen from some Chomsky video)
Shouldn't the device be the judge of that?
It’s a cyberpunk anime where society uses a system called the Sibyl System to constantly scan people’s mental states and “crime potential” (their Psycho-Pass).
People can be arrested before they’ve done anything - just because the system picked up certain signals from them.
Very, very interesting idea
If, one day someone can make a small, cheap device that can do the job of a fMRI it would be more world changing than you can imagine. If you had easy access to realtime data about what is going on in your brain, there is evidence to suggest that you can learn to influence the data and literally change your own mind.
There are also developments in ultra-low-field fMRI (<0.1T) which use permanent magnets, which are estimated to retail in the five figure range, however it's more for structural usage (can identify a tumour or stroke progression).
What you are saying sounds like being able to control your own heart rate if you see it on a monitor. Maybe combining low resolution fMRI with models trained on higher resolution data, could give you enough visibility that you could learn how to activate other areas of the brain that you wouldn't normally use for tasks.
"There is a real you. Unfortunately, it's you."
Sounds like Libet's Delay and all that. Conscious awareness is just a documentary covering something that has been decided some half a second ago.
Consider that heroin used to be legal. We might have a dozen such technologies that we know are hazardous but have yet to kill enough people to force the issue.
It's a very interesting quirk of a immensely useful device for those that need it, but it's not an ethical dilemma.
I for one am sick and tired of these so-called ethicists who's only work appear to be so stir up outrage over nothing holding back medicinal progress.
Similar disingenuous articles appeared when stem-cell research was new, and still do from time to time. Saving lives and improving life for the least fortunate is not an ethical dilemma, it's an unequivocally good thing.
Quit the concern trolling nature.com, you're supposed to be better than that
The problem with humanity is some people pick up the hammer and build a house while others will crack your head open with it and eat the pink gooey insides. The discussion of technology should be able to withstand the good and bad points of its conception.
True, but that's a poor excuse to make up hypothetical problems that don't even exist. Tying it into the current craze about data privacy is a bit too transparent imo.
The journalists should do better, the article makes a downright dishonest interpretation of the issue by shoehorning it into a lens of a long ongoing controversy about data privacy that is in no way applicable to the technology they're discussing.
The privacy issue is not only hypothetical, but far enough off in the future that it'd be a better fit for a sci-fi novel then nature.com reporting.
The problem is not discussing the downsides, the problem is doing it dishonestly, as the article does.
the miracle that is humans and humanity came about through millennia of unrestrained fertility and lots of sex℠ producing many babies, most of whom didn't survive to adulthood. Insects and fish still do this on a grand scale. It's where we came from and who we are, and that's Not Bad™ and I think it's unequivocally a bad thing that we keep thinking we know better and interfering with it. we are failing miserably to propel our species forward for the future. "survival of the fittest" is a mistake, it's destruction of the no-longer-adequate that works the magic.
(pretty proud of myself for realizing after I put in the tm that I could go back and put in a service mark too)
I'll believe you actually hold this opinion after you die from an entirely curable decease. Until then I'll assume you actually think we do know better and that our interfering with, for example, bacterial infections, is actually a good thing.
Not to tell you what you think, but this is one of those opinions people seem to think they hold until it's phrased in a way that demonstrates the cost of the idea to them.
Just the fact that you survived long enough to type this comment into HN tells me it's highly unlikely medicine hasn't saved your life at least once already. And if it hasn't, it will.
I can count 5 times of the top of my head that I would've died if not for modern medicine (usually antibiotics, but also one life saving surgery). I would've been rendered a cripple a few times over too!
what does that have to do with anything?
narcissism and self importance are adaptive features we use to propel our genes into the future but that doesn't mean anybody else should care or help us, they are trying to look out for their own offspring
I'm talking about public policy, not your or my personal feelings. I think public policy should focus on creating the strongest possible tribe for the future.
when I hear people here whinge on the daily about how health care should be free for everybody, I yawn. No matter what line item something is in the budget, I want to hear "what do you think we should say "no" to? We can never buy all of everything, there are always tradeoffs, and if you aren't willing to announce your tradeoffs, you aren't serious.
what feelgood items will you say no to?
oh, and your other challenge, you would have been closer to the mark if you said "fsckboy, the only way you could hold that opinion is if you have been so remarkably healthy and accident free your entire life that it doesn't even make sense to you the degree to which other people want to endlessly talk about their personal and family frailties" nobody has ever needed to save my life, and i'm old now so nobody should care. Neither one of my parents ever took a single day off of work for illness.
Hold on, I dropped my Nobel Prize for lobotomy on the ground somewhere... going to need some help looking for it...
On second thought, maybe it's better off not being found.
I love seeing the advancements still, don’t get me wrong, but in the current data, advertising, and attention economies under Capitalism? No fucking way that shit is ending up in my head.
> Smith’s BCI system, implanted as part of a clinical trial, trained on her brain signals as she imagined playing the keyboard. That learning enabled the system to detect her intention to play hundreds of milliseconds before she consciously attempted to do so
There are some serious problems lurking in the narrative here.
Let's look at it this way: they trained a statistical model on all of the brain patterns that happen when the patient performs a specific task. Next, the model was presented with the same brain pattern. When would you expect the model to complete the pattern? As soon as it recognizes the pattern, of course!
> That learning enabled the system to detect her intention to play hundreds of milliseconds before she consciously attempted to do so
There are two overconfident assumptions at play here:
1. Researchers can accurately measure the moment she "consciously attempted" to perform the pretrained task.
2. Whatever brain patterns that happened before this arbitrary moment are relevant to the patient's intention.
There's supposed to be a contradiction here: The first assumption is correct, and the second assumption is also correct. Therefore, the second assumption does not invalidate the first assumption. How? Because the circumstances of the second assumption are a special thing called "precognition"... Tautological nonsense.
Not only do these assumptions blatantly contradict each other, they are totally irrelevant to the model itself. The BCI system was trained on her brain signals during the entirety of her performance. It did not model "her intention" as anything distinct from the rest of the session. It modeled the performance. How can we know that when the patient begins a totally different task, that the model won't just "play the piano" like it was trained to? Oh wait, we do know:
> But there was a twist. For Smith, it seemed as if the piano played itself. “It felt like the keys just automatically hit themselves without me thinking about it,” she said at the time. “It just seemed like it knew the tune, and it just did it on its own.”
So the model is not responding to her intention. That's supposed to support your hypothesis how?
---
These are exactly the kind of narrative problems I expect to find any "AI" research buried in. How did we get here? I'll give you a hint:
> Along the way, he says, AI will continue to improve decoding capabilities and change how these systems serve their users.
This is the fundamental miscommunication. Statistical models are not decoders. Decoding is a symbolic task. The entire point of a statistical model is to overcome the limitations of symbolic logic by not doing symbolic logic.
By failing to recognize this distinction, the narrative leads us right to all the familiar tropes:
LLMs are able to perform logical deduction. They solve riddles, math problems, and find bugs in your code. Until they don't, that is. When an LLM performs any of these tasks wrong, that's simply a case of "hallucination". The more practice it gets, the fewer instances of hallucination, right? We are just hitting the current "limitation".
This entire story is predicated on the premise that statistical models somehow perform symbolic logic. They don't. The only thing a statistical model does is hallucinate. So how can it finish your math homework? It's seen enough examples to statistically stumble into the right answer. That's it. No logic, just weighted chance.
Correlation is not causation. Statistical relevance is not symbolic logic. If we fail to recognize the latter distinction, we are doomed to be ignorant of the former.