Work on using AI to brain activity into speech | Hacker News Reader

Work on using AI to brain activity into speech

sciencemag.org

72 pointsOsiris3012 comments

After working on the same task ~3 years back I will say: Be skeptical about results. Neural activity shifts over the course of days, and experiments without long term recordings will produce results which look a lot better than what can be obtained in realistic scenarios. As the article also mentions, there is a sizable question about the differences in signals when vocalization is not possible. I've seen some of that data, and in my opinion we're still a good whiles away from being able to use that data in any practical sense.

Iam very interested in brain computer interfaces. It would be great if you can share any of your work here..

Poster from extracting speech articulation events using local field potentials (electrodes inserted into brain matter) http://fundamental-code.com/tmp/2015-neurosig.pdf

How does it compare with what https://www.ctrl-labs.com/ are doing? It seems that they get the intent just from having a device attached to the arm, so is it really necessary to attach the electrodes to the brain? Because if the person is reading silently you can still get the signal going from the brain to the muscles in the throat/mouth, can it be just attached to the neck?

Typically the end target for applications like this are for patients who have some impairment which prevents normal vocalization which would mean that the EMG around the throat/mouth would not be reliably available.

With these kinds of MMIs, I often wonder if it wouldn't be better to train the mind to the machine instead of the other way around. What I mean is, have the machine convert certain brain activity into phonemes or some other primitive and allow the human mind to do what it is already pretty good at: learn to use a tool.

Since I'm not an expert in this domain, I assume there are reasons this isn't how it is done. Is anyone aware of what they are?

To a degree there is some learning on both sides. I don't know if I've seen it mentioned in detail in papers that I've read, but it was a general tone at a few confs when dealing with coarse motor functionality.

As per why the same approach is not well suited for speech, one (of many) components is the data bandwidth you need for speech to happen at a reasonable rate. For many applications the ML involved in the brain computer interface is only able to extract a few bits per second. It works for applications where the output is coarse (e.g. up/down/left/right) or where large delays are acceptable, but it's not the best fit for speech.

> extract a few bits per second

That's surprising. I thought it would be a lot higher than 3.3 bits for 10 fingers (probably not worth considering in this context).

To me, this is the only hope. Herf says it is one approach at the end of the article.

> If they can hear the computer's speech interpretation in real time, they may be able to adjust their thoughts to get the result they want.

I wonder if thought adjustments are intuitive or tricky.

The eeriness of the second audio clip reminds me of the first recorded voice by Edouard-Leon Scott de Martinville:

https://www.youtube.com/watch?v=q7Gi6j4w3DY

Stephen hawking: AAAAHHHH you got be kidding me

This had me legitimately laughing out loud. Thank you ^_^

Reply on news.ycombinator.com