Musician Who Lost His Arm Plays Piano Again with AI Prosthesis | Hacker News Reader

Musician Who Lost His Arm Plays Piano Again with AI Prosthesis

blogs.nvidia.com

88 pointsbcaulfield24 comments

This reminds me of Paul Wittgenstein (brother of philospher Ludwig)[1]. He was a concert pianist who lost his right arm in WWI and the commissioned a bunch of piano music playable with only the left hand, including works by still-famous names like Ravel and Prokofiev. My personal favorite is Korngold's [2].

More recently, there is Leon Flescher, who was a highly-renowned pianist until he lost the use of his right hand in the 60s. Amazingly, he got it back 40 years later and has enjoyed recent success again as a two-handed pianist.

[1] https://en.wikipedia.org/wiki/Paul_Wittgenstein [2] https://www.youtube.com/watch?v=bGulyl8bzgQ

Reminds me a bit of the drummer for Def Leopard, learned to drum one handed after car crash left him.with amputated arm.

A similar case is the Finland-resident Japanese pianist Izumi Tateno, who lost the use of the right half of his body after a stroke and then began giving concerts solely with his left hand.

Wow, that's really interesting. Thanks for sharing this info.

Some famous (ish?) modified versions of Chopin's Etudes were written by Godowsky. His mission appears to have been to make Chopin's hardest pieces just a bit harder, including the Revolutionary for the left hand only: https://youtu.be/X2nMUwdh1Wk?t=2m41s

Woah

I had the pleasure of seeing Nicolas McCarthy speak and play live a couple years ago.

He was born with only a right hand, and fought the odds to start learning piano at age 14. He eventually made it into and graduated from the Royal Academy of Music in London, their first one-handed pianist.

He plays a variety of left-hand-only repertoire. His playing is amazing, and his story inspirational.

http://nicholasmccarthy.co.uk

Typically these “futuristic” prosthetics are still relying on myoelectric sensing and a lot of wishful thinking. It’s nice to see some meaningful progress in more detailed sensing technologies. This actually seems promising.

This was a hot thing a few years ago. These guys still seem to be around:

https://techcrunch.com/2013/06/05/thalmic-labs-raises-14-5m-...

https://www.myo.com

In the realm of spinal cord (and nerve repair) this work also struck me as extremely promising: https://www.twu.ca/news-events/news/twu-graduate-creates-nan...

Once these are working well will factory workers be able to attach a third and fourth arm for extra productivity?

The brain has a couple of regions (the somatosensory context, the primary and supplementary motor cortex) with maps of the body. You'd need to add the limbs to those maps. Brains have enough plasticity to re-allocate the area responsible for limbs or fingers by a minor amount, but nowhere near enough to add a new limb, at least in adulthood.

I'm not so sure that's true.

People who use a tool for a long time report the tool feeling like an extension of their body.

Also, try this thought experiment - try to flap your wings. I know you don't have any, just try to activate them anyway. Do you sense yourself trying to move "something"? Your brain clearly is able to map anything it wants.

For the technology specifically covered in this article, all they are doing is reading the muscle activation of existing muscles, and then mapping to the prosthetic's actuators. The "twist" in this story is that sensing is done with ultrasound instead of EMG (better signal to noise if you can throw enough compute at it), and I believe some cool deep learning tricks to map sets of muscle actuations to prosthetic motion "better".

In order for this approach to work, a nervous signal has to travel out of the brain, and at least into the brain stem. In these cases, they actually have to get all the way to a giant amplifier (your muscle) in order to detect it.

When your talking about your brain imagining to flap your wings, unless your brain has mapped your swings to your lats, traps and delts, you're not sending any actual motor signals out of your brain and we can't read it. If it is mapped to your lats, traps and delts, then you're actually moving those muscles right now. If we used a system where we tried using your lats, traps and delts as points to control prothestic wings, then you run into the problem of what do you with your wings when you're not trying to use those muscles as a wing controller. In other words, you'll get a modal system... which are tricky. You'd basically have to switch off parts of 'baseline human' functionality at a time to get your 'human+' stuff.

We have useless muscles in outer ears though. Maybe it is possible to repurpose them.

You mean like this? https://www.youtube.com/watch?v=8bj_-9t89Bk#t=04m12s

That just means we'll have to attach third and fourth arms to children bred for labor.

What about programmers? Now you can have more than one home row.

Attach, maybe not...

But why not remote-control multiple sets of arms doing exactly what the primary pair are doing?

It is progress, and it is very impressive but the title is a bit misleading.

I think there is still a gap between what is being demonstrated to fully "play piano again", in the context of playing piano professionally as a musician.

For composition it can be still very valuable.

What really frustrates me when I watch this kind of demonstrations with bionics is the delay and inaccuracy between desired action and the real effect. I know it's not that simple, but from my perspective since the desired actions are transmitted through our nerves at 100 m/s and we can send an message thousands of kms away in milliseconds, I think there is always space for improvement. I am curious which is the real bottleneck in all the process.

As you can see in the video https://www.youtube.com/watch?v=HjW1kIt5iQg there is some latency. Weinberg and Barnes are working to fix that, but I don't know details. Weinberg said he wants to gather more data from more users so the arm is more useful to more people.

But why should one rely on the termination of the nerves instead of the implating an electrode on the main cortex which controls the motor activity? Can't they translate directly the actions from the energy flow at that level?

how hard would it be to translate the movements to midi? I'm guessing there would be some way to capture the signals the prosthesis and characterise the mapping from movement to midi.

Reply on news.ycombinator.com