Vibrating vests translate music for deaf concertgoers (2023) | Hacker News Reader

Vibrating vests translate music for deaf concertgoers (2023)

techxplore.com

54 pointssargstuff35 comments

This guy does a lot of great work on sensory substitution. [1] He runs a company that makes tools that help with tinnitus and other chronic issues/disabilities, and he writes books about the brain.

1: https://www.youtube.com/watch?v=4c1lqFXHvqI

It's fascinating how plug & play our brain is. Just consistently supply information to it and voila, you have a new sense.

One day we may have a marketplace where you can purchase a sense of north or your spouse's body temperature, it will be interesting and weird.

There are languages without the terms "Left, Right, Front, and Behind." These cultures use "North, South, East, and West" their whole lives no matter if they are in a room or in the woods. The result is a compass brain. They always know how they are oriented.

https://www.nytimes.com/2010/08/29/magazine/29language-t.htm...

Others do not have language for absolute directions. The Murrinhpatha in Australia. https://www.degruyter.com/document/doi/10.1515/opli-2016-000... I think the Pirahã may only use directions relative to the river

I’m curious how or if that affects physical training, like gymnastics or martial arts.

Here's a long passage from the article that sheds a little bit of light on it. My assumption before reading it was that even if they don't use egocentric coordinates, they would still have a concept of "left hand" and "right hand" as permanent objects, since they're not identical (people usually have a dominant hand). Now I'm not so sure!

> When these peculiarities of Guugu Yimithirr were uncovered, they inspired a large-scale research project into the language of space. And as it happens, Guugu Yimithirr is not a freak occurrence; languages that rely primarily on geographical coordinates are scattered around the world, from Polynesia to Mexico, from Namibia to Bali. For us, it might seem the height of absurdity for a dance teacher to say, “Now raise your north hand and move your south leg eastward.” But the joke would be lost on some: the Canadian-American musicologist Colin McPhee, who spent several years on Bali in the 1930s, recalls a young boy who showed great talent for dancing. As there was no instructor in the child’s village, McPhee arranged for him to stay with a teacher in a different village. But when he came to check on the boy’s progress after a few days, he found the boy dejected and the teacher exasperated. It was impossible to teach the boy anything, because he simply did not understand any of the instructions. When told to take “three steps east” or “bend southwest,” he didn’t know what to do. The boy would not have had the least trouble with these directions in his own village, but because the landscape in the new village was entirely unfamiliar, he became disoriented and confused. Why didn’t the teacher use different instructions? He would probably have replied that saying “take three steps forward” or “bend backward” would be the height of absurdity.

No, they don't magically have some sense of north. They're just used to using it so much that they have a lot of points of reference to know which direction is which no matter where they are.

Not necessarily just a sense of reference, but also a practice of always keeping in mind what direction one is when changing direction (mentally summing up the deltas in direction).

I do this when driving - I always know what direction I'm headed, even in an unfamiliar place (unless there are a ton of non-right-angle turns that confuse me, and then I have to check a map).

Blocking a punch or doing a kata also has no concept of north. It's usually bilateral. So if you don't have left or right... Do you pretend your face is north and instruct that way?

I'm not sure what you mean. Whichever way you happen to be facing, you can use an earth-relative coordinate system to disambiguate limbs rather than a person-relative one. I guess don't practice martial arts while standing on the north or south pole and you'll be fine.

How do you explain throwing or joint-locking an opponent to someone who does not believe they have a right or a left arm? Both of these actions can involve turning around. Anywhere from 45º to 360º.

It's just a succinct abstraction.

The sense of north is (was) already a thing. It kind of sucked though, and injecting a HUD into the front of a night vision tube is a lot more practical.

I've seen a few hacky projects (e. g. https://www.carlosterminel.com/wearable-compass) over the years about haptic belts, and I've been wanting to try one of these, but I've never seen them commercialized. Have you tried them? Otherwise, could you please describe why the sense of north "kind of sucked"?

In my experience, having a flagship Android phone with unreliable compass such that I cannot even get my bearings when leaving a subway station, is extremely annoying. Such a belt would seem more useful to me.

I wonder if they would need a vest at the last hard rock/metal concert I went to.

Sheesh, I think they almost blew off my glasses with one of their drum/bass hits that night.

Its interesting that now we have tech to let the deaf participate in concert events while at the same time you have folks like me cramming earplugs as deep as they'll go to be able to enjoy them comfortably.

I used to work for the firm that sells "The Vest", which helps people with cystic fibrosis and other lung problems clear their airways (I worked on a different product).

It rapidly inflates/deflates to vibrate the chest cavity (not unlike attending a metal concert and standing in front of the speaker stack) to dislodge mucus. Using it can significantly increase the life span of people who haven't/can't have a lung transplant.

https://www.youtube.com/watch?v=2rZbDcPiJv0

Hehe, I was about to say as well: We have that in metal. It's called a bass and kickdrums.

> I wonder if they would need a vest at the last hard rock/metal concert I went to.

I know quite a few people hard of hearing (either from age, birth or .. well, not entirely responsible consumption of loud music) who love metal concerts exactly for that.

It's weird to people who haven't experienced it. Think about it that way: There are different qualities to enjoy the music. At first, you have shitty laptop speakers which... don't have a lower end, at all. Then you get speakers with a lower end and suddenly the song gets a whole new dimension of kick to it, as now the bass and the headbanging kickdrum really start happening.

And then you go to a concert with a good sound system and that kick drum or those blast beats become a full body experience. Even if you wear plugs - and you totally should wear quality ear plugs[1] - you can feel the rhythm section. Lemmy wasn't joking with the line "Here comes the bass - thunder in the guts". Some of the good rythm sections feel like there is an actual wall of sound hitting you.

1 - I'd recommend everyone enjoying loud music to spend 20-30 euros on some good senner plugs. Mine have lasted for years already - the only way I see needing a new one would be losing one. It's really interesting, because they just cut out a lot of sound pressure from the lower ends without cutting out melody or vocals. You can talk to people with them in almost normally.

I've even started wearing them during a company event some time ago. There were a million people talking in the same room and it made it easier to understand the person I was talking to, because the plugs took out the insane background murmuring.

And those are the plugs professional sound guys use, so you get closer to how the mix is supposed to sound :)

The show which you could feel the music the most (at least that I have been to) is GWAR.

You'd think they wouldn't have great audio guys working on their shows, but you'd be wrong. The drums, the guitars, the bass, it all rumbles through you in the best way.

If you're deaf, go check out a GWAR show! Bring a rain poncho, if you're not familiar with the band.

I spent a couple years at Bergen Community College in New Jersey which has a great deaf/asl program. I made friends with some dead kids and they took me dancing. The music was so loud and bassy that you could feel it in your teeth. Anybody who wasn't deaf would have had a tough time there.

Not even just concerts. I went to a movie theater recently and could not believe how deafeningly loud it was. This was for a stupid comedy film, not a boomy action flick.

I just saw Dune part 2 at the local IMAX. It was crazy loud. My SO wore earplugs throughout, and felt that was a comfortable level...

The sad part was they didn't really use the dynamic range, there was so much loud music and sounds so often, that the really epic moments didn't stand out in that sense.

I've never understood this kind of thing. My immediate impression - as someone with hearing who has never tried one of these vests - is not that deaf people are being included, but rather they're being given an entirely separate experience. It's almost like if I went to a Taylor Swift concert but brought my headphones and listened to Bon Jovi the whole time.

How do you "translate" music into vibrations while preserving the feeling created by the original work? Do these vests actually create a similar experience for deaf people or are they just something novel to occupy themselves with while everyone else is listening to the music?

Not trying to be cynical here. I'm genuinely curious if anyone can speak to this from experience.

It's not well-explained in the article but there's a separate artist "playing" the haptic vests via programming, not unlike how a lighting engineer programs the synchronized light shows for a big touring act.

Instead of a DMX controller choosing entire scenes that control banks of lights in unison, or narrowing down to individual banks of lights to fine-tune things on the fly with faders, the haptic vests use similar commands and controllers to program different intensities and patterns of vibrations across the sub-regions within the vest.

The haptic engineer/artist usually has packs of go-to scenes that work well and provide distinct but complimentary effects from each other scene (just like the lighting engineer), and may have an entire effect chain pre-programmed for a song or mix, or may be choosing and mixing the haptic scenes on-the-fly as they listen to the audio.

This NPR article has a better description: https://www.npr.org/2023/07/17/1186173942/vibrating-haptic-s...

> Do these vests actually create a similar experience for deaf people

I don't think there's any technology that could make a deaf person feel the difference between major and minor or appreciate counterpoint or whatever, but I don't think that's a standard anybody would expect. Accessibility doesn't mean giving everyone identical experiences because it's impossible.

But do you and I even hear the same thing when we listen to the same music? Probably not, or else we'd have the same opinion of it. Creating an experience we all can enjoy is worth it, even if it's never going to be the same.

It reminds me of Mr Holland's Opus.

> But do you and I even hear the same thing when we listen to the same music?

I say "yes". We can share an experience but come to different conclusions about it.

> Creating an experience we all can enjoy is worth it, even if it's never going to be the same.

This makes me think of vegan food. I'm not vegan but there are many vegan dishes I thoroughly enjoy. Despite that, I'm consistently disappointed by vegan adaptations of non-vegan dishes. The best vegan dishes IMO are the ones that don't pretend to be something they're not.

How do you know we hear the same thing? We likely don't hear the same frequencies, as these things diminish with age.

> How do you "translate" music into vibrations

Is there anything to translate? Music is vibrations to begin with.

Yes but the experience of listening to music is nothing like the experience of someone vibrating your chest. Our brains treat those sensations as very different things.

Our brains and other sensory apparatus are extremely adaptable. If one doesn't know anything about the sensation of sound, they can learn to substitute it without it actually being a substitute to them.

They may not be able to feel the difference between notes, especially in the higher frequencies, but they can still feel the percussion, bass, etc. They can still dance to the same beat as everyone else even if their eyes are closed. They can feel the overall intensity.

Reminds me of Cutaneous Grooves (2003)

https://www.tandfonline.com/doi/abs/10.1076/jnmr.32.4.369.18...

Although that idea was more about using haptics as a medium to compose explicitly for rather than translate existing music to.

This article[1] mentions a composer who "composed primarily for the vibrations and included the music after the initial layout of the haptic composition." We used to put on whatever 3D glasses the theater was providing for big new 3D movies; why not don a vest to listen to Deadmau5?

[1] https://www.classicfm.com/discover-music/live-music-deaf-aud...

I was in a Kickstarter for https://www.woojer.com/ and it was kind of fun. At one point, I had that hooked up to some music, my Oculus Quest hooked up to 3D video. It was fun enough, but a lot of harness and cable stuff so I didn't end up using it that much.

I've had a Subpac for years, never really considered the applications for the deaf. But it would be great. Really, seriously elevates music. But I imagine you get a lot of the same feeling already during concerts.

Url changed from https://old.reddit.com/r/gadgets/comments/15b1d3z/vibrating_..., which points to this.

Reply on news.ycombinator.com