A soft, wearable brain–machine interface
spectrum.ieee.org
spectrum.ieee.org
Like... setting aside all the things that, given enough time/effort, we'll reach some useful maximum. I'm told that with ML (eg: image classification) we'll eventually train the systems enough that they'll do a pretty amazing job.
So is it converting analog brain signals to digital? Is the rate of data transmission even relevant here? What would happen if we had enough "brain data" from a single person to saturate a 10 gigabit network? Do we have the software to do anything meaningful with that?
Upgrade to USB 4 :)
My lab has equipment that collects data from 288 electrodes in the brain, each at 16 bits/sample x 30,000 samples/sec. This works out to 140 mbps, not counting overhead or other types of data we collect, and it’s not abnormally large. If we had a research question that required it, the vendor sells versions with up to 512 channels—-and you can gang them together for even more.
However, this probably isn’t the direction BCI is headed. A lot of the signal is redundant. Some of this is because nearby neurons tend to do very similar things, and some of it is because signals propagate pretty well through the brain so electrodes pick up the same signal in different places. As a result, most non-research applications don’t need all of that data and it’s increasingly possible to do some preprocessing right at the brain: folks at Imperial have made all kinds of cool ASICs that extract spikes.
There are however a huge number of slow signals running in parallel. In total these would add up to a huge bandwidth if constrained on a single channel.
To put it another way, the scull is almost entirely resistive. And you can't make a low-pass-filter with just resistors.
There's no obvious way forward with any of these that produces what you or I might consider a true brain machine interface. We don't have the tech AND we don't understand the brain enough.
Fortunately you don't need a brain-reading device to produce something useful, just like you don't need a teraflop computer to go to the moon. I've written recently about an EEG helmet that can be used by profoundly disabled folks to navigate a UI, type, and so on, and that doesn't require a precise signal at all. So I think what you'll find is while the Musks of the world are chasing a sci-fi dream of what they think the technology ought to be, most of the utility will come out of using what it's actually capable of in a smart and compassionate way.
If we're ever to achieve any measure of "immortality", BCI is probably the only way.
We could clone monoclonal, brainless humans with universal HLA haplotypes for spare parts assuming we could get past the religious/ick factor. Beyond just organ harvesting, a full body (ie head) transplant could rejuvenate the immune system and potentially reverse many of the effects of aging assuming we could get past central tolerance. (Maybe not an issue with immunosuppressants or monoclonal lines without B/T cells, but that sucks and I think it can be optimized.)
This would be the best next step in increasing human lifespan dramatically. It wouldn't save you from physical accident-induced death or irreversible brain atrophy, though.
Completely sci-fi conjecture:
If we could build clusters of machines capable of running the same distributed work that the human brain does in real time, and if we could get enough signal out of the brain through much more advanced (and invasive) instruments, we might be able to model a subset of a person's memories and run them at real time.
If instead of just copying signal, perhaps we could leave a human hooked up and "amplify" their capacity for thinking by supplementing their brain with a computer. We might be able to copy memories into the new computerized system while the person is still alive and thinking. If the way consciousness works is amenable to such a process, we might be able to perform a one way "move" operation. Essentially digitizing a person as a destructive operation, and killing the body when the process is complete.
Of course we're not anywhere close to anything like this. It's all hypothetical and not at all close to the science.
Cloning humans though: entirely within our capacity. We'd see incredible medical benefits in doing it, including substantial lifespan increases if we routinely replace older parts sourced from clones. People are just too religious.
(Monoclonal brainless humans would lack a consciousness. They're not much different from plants in that respect. Entirely ethical to use for parts and experiments.)
Do we think any "mad scientist" types doing this, or is it just easier to use condemed prisoners?
It is simpler, cleaner, and less prone to malfeasance and corruption to limit organ harvesting to registered consenting individuals and lab grown tissue, where a chain of custody for the tissue can be established.
I'm not sure this is strictly a religious matter. It could be rather deeply rooted into the perception of humanity, the human "self" that we have, and the reciprocity of it towards others: how would you make sure that a "grown" clone is not "someone" already?
"Well Mr Henderson, that's it. We've scanned every synapse and uploaded your consciousness. Now if you'll just step into the vaporization chamber..."
An approach for this is training models that have expectations based on the stimuli themselves; e.g., to find neural resonance with different frequencies in the stimulus.
Signals are only somewhat local - you get increased activity in this or that region and can correlate it with what happens in the brain - like power analysis attacks on crypto chips - the algorithm has to recover the data from the limited, noisy signal we are able to pickup.
You can't just put electrode in every single neuron to read its state like you can't put an electrode into every single piece of conductive metal on a chip - it would be to tight to fit and everything would stop working.
More realistically it requires access to every single element inside those neurons.
Otherwise you're just poking at things in concrete gloves.
Not only is there no technology for this, there isn't even a foundation which could be developed to create the technology.
Extracting a control signal with few degrees of freedom from an EEG signal is doable right now. A reasonably motivated undergrad with a few hundred bucks of OpenBCI gear should be able to get “MindPong” up and running pretty quickly. But…it’ll be a little janky: the signal quality won’t be great, especially if you’re out in the real world, trying to do normal stuff at the same time. The content of that signal is also limited, in part due to the skull and scalp that sit between your electrode in the brain. This also make it difficult but, surprisingly, not impossible to perturb the brain with electric current (tDCS, tACS), magnets (TMS), or other techniques.
What about putting something inside the brain itself? We know information is likely much more accessible without the skull/scalp filtering, but we don’t completely understand how information is represented in the brain, how to modify those representations, or even really how to get the raw data out: most neural implants have a pretty short lifetime before they’re ruined by the immune system, mechanical strain, etc. We’re not totally ignorant: there’s been some amazing progress decoding motor and speech intentions and, after a 20+ year hiatus, cool new electrode technology (Pandromics, Neuralink, etc) but there’s a lot to be discovered and invented.
To sum up, we have some fairly crude stuff working now, but it’s looking for a killer app, especially in humans. Building something more like a movie requires work on many different fronts, ranging from materials science to build the electrodes to neuro/ML to understand what those electrodes see.
I always see futurology/inspiration porn articles about impressive demos of this stuff, when is it hitting the market? It doesn't have to be the next iPhone, it just has to let people control their mobility or communicate.
Accessibility gear needs to be robust, but a lot of the BCIs, especially the noninvasive ones, tend to be a bit janky. As a result, you can zip around in VR once the experienced lab tech sets you up, but maybe don’t have the DOF to control a real wheelchair or a system that disabled people can set up themselves. Some of this is probably just need some good systems engineering, but there are some legit technical challenges too.
I think this will start to change soon: there’s a lot of money flowing into neurotech and hopefully, some of it will end up with people who are more serious than hype-y. (If nothing else, I’d like a job :-))
I hope it changes too. I have a progressive disability and the lack of innovation/competition in assistive devices is starting to make me nervous as I start to need more.
I think it's quite a bit earlier than that. If you get motor imagery working, which is the most reliable signal (your brain is electrically very active when visualizing motor tasks), your accuracy rate is still crazy low. And it doesn't generalize across people since our brains are all pretty different. Some people can't make the EEG work.
I’d say it’s a bit like mid-90s speech recognition: it works well enough to be intriguing, but the hassle and inaccuracy make people favor other alternatives. I’d take an eye tracker over an EEG speller, for example, if I had ALS.
Some of this is due to hard technical problems, like building better electrodes or squeezing more information out of the signals, but I think there’s a lot of low-hanging fruit too. For example, many spellers don’t include language models, which, as someone originally trained in speech recognition, absolutely blows my mind.
I'm with you on "I'd prefer an eye tracker." Modern advances in eye tracking (saccade-based stuff!) are pretty sick, and make the technology usable for long periods of time. You have higher bandwidth input than motor imagery, and don't torture the eye like P300.
She is building the state of the art brain scanner and neuron read/write with 50 chips
Imagine the feeling of going to reach for something, and accidentally just envisioning moving your arm. Might be like "stepping into air" when you're almost asleep.
Time to go outside for a bit.
Sometimes if I practice difficult scales (those with novel movements for weak fingers e.g. left pinky and ring) on the piano before typing on a keyboard, I'll mistype more frequently.
I wonder if this type and similar phenomena is also related: https://en.wikipedia.org/wiki/Phantom_vibration_syndrome
EDIT: This device is only penetrating the scalp, but other devices like Neuralink penetrate into the brain.
Motion is a problem for things that are inserted into the brain, but people have been getting DBS and sEEG implants for 30 years, and it’s manageable, though not totally solved.
- proven usefulness for general computer tasks
- must be relatively easy and quick to set up
- cost must be reasonable, if expensive - anything over something like $2,000 is going to severely limit your customer pool. A semi-automated manufacturing line will help with this
- for good measure, a crack marketing team
Another consideration is that the BCI industry is moving insanely fast. By the time you set up a production line and start mass-producing, there might be news about a novel much-better process that overshadows your product and destroys sales. This is effectively what happened with VR in 2012-2018 when everything was super expensive; only as of recent can you recommend an Index or Quest 2 to people without fearing a huge leap in VR quality to be coming in the near future.
Eh, not necessarily, early BCIs if actually useful will likely be targeted towards people with disabilities and thus funded through private or socialized health insurance. Once the hardware undergoes commoditization consumer grade BCI will become inevitable with low price points. VR evolved slower because there was no middleman like insurance buying hardware for their customer pool (the counterpoint being Apple Watch which you have been able to get at a huge discount with some insurers).
"A research team led by a 2018 BBRF Young Investigator, Sung Il Park, Ph.D., reports that it has developed and tested a new technology enabling unprecedented exploration of nerve-cell function inside organs of the body outside of the brain.
The new technology, called optoelectronics, uses tiny wireless implantable devices to manipulate the activity of individual nerve cells in the organs of awake, freely moving animals. This makes possible experiments with the power to reveal the specific (and often multiple) functions of different kinds of nerve cells in the body's periphery."
https://www.bbrfoundation.org/content/new-technology-enables...
With say the traditional EEG electrodes, they seem to cover all over the head (some with say around 128 electrodes).
I understand that they'd get a higher quality signal, but is the fact they're only using electrodes over one area of the scalp a disadvantage?
The Muse hardware is seriously inhibited by the same user-exploitative trends we see everywhere, accompanied by meaningless promises. Many researchers used the Muse SDK and API until they discontinued it in 2019 [3] in favor of Muse Direct - another way to force users to feed Muse their data and lock them into a subscription.
Unsurprisingly, this was also abandoned [4] and they claim "We're working on a solution that allows you to collect raw EEG data using the free to download" app, which would still allow Muse complete control over your authentication, use and data. Ugh.
At least the community has reverse-engineered the LSL protocol and built a Python package [5].
Flowtime claimed that it offers a closed API, as they don't have the resources to maintain an open API [6]. I remember somewhere they claimed it was on their roadmap, but I searched again today and couldn't find any support of their claim.
BrainBit [7] and NextMind [8] seem to provide more robust developer tools, although I have no experience. Naturally, each of these runs more than double the ~$200 cost of the Muse and Flowtime devices.
I have no formal neurological education but I strongly believe non-invasive EEGs, combined with novel, gamified training techniques coupled with machine learning will usher in a new era of digital interactions. Like the transition from tactile smartphone keypads to the eventually-ubiquitous full touchscreen, I expect many design iterations in both hardware and software will produce utterly fascinating products.
I can't wait to get excited again for a product launch.
[1] https://choosemuse.com/muse-2/ [2] https://www.indiegogo.com/projects/flowtime-biosensing-medit... [3] https://github.com/sccn/labstreaminglayer/issues/30 [4] https://choosemuse.force.com/s/article/Muse-Direct-Subscript... [5] https://github.com/alexandrebarachant/muse-lsl [6] https://www.kickstarter.com/projects/987756376/flowtime-bios... [7] https://sdk.brainbit.com/ [8] https://www.next-mind.com/developer/
ex. a simple "yes / no" detector would be meh, pointer control would be amazing, some sort of mood detector/brain trainer type apps would be _awesome_ since it'd be producing value for me