Hacker News Reader

Comment by fudged71 | Hacker News Reader

Artlav, this is an incredible project!

I have an idea for you. You mention that you want to use a webcam so that you can "overlay" this information on top of an image.

What if you could send it directly to your brain?

There is a technology called Brainport[0], which is effectively a 16x16 grid of electrodes small enough to place on your tongue. By feeding electrical impulses through these electrodes, you stimulate the nerves on your tongue. After two minutes of a "champagn bubbling" sensation, your brain rewires the senses from your tongue to your visual cortex and you can "sense" any input.

This technology has been used to give sight to the blind, balance to those with inner ear problems, and sonar to NAVY seals.

It would be amazing if you were the first person in the world to truly "see" sound.

[0] https://en.wikipedia.org/wiki/Brainport

View on HN

BrainPort sounds extremely cool. How does the brain decide to interpret them as visual input, though?

I'm pretty sure that would be mechanically impossible - the brain is very plastic, but touch-into-sight seems a little far for adult learning. Instead, you could expect this to feel more like acquiring a new sense - the experience reported by those who have used pager vibrators to give their body a compass[1].

[1]http://www.thinkgeek.com/product/f358/ (sorry, I couldn't find a better source - but the product description explains it)

one of those projects is feelSpace[1] which is a belt with vibrators giving you either a new compass sense, or tells you where to go (as a navigation system).

[1] https://www.indiegogo.com/projects/feelspace-follow-your-gut...

Thanks for the link. I have wanted one of these belts for years! But I can't believe how expensive it is.

Whilst a commercial offering is probably quite expensive, it strikes me as the sort of thing you could do quite cheaply and easily with an arduino.

I've actually been collecting the vibration motors from disposable electric toothbrushes to build my own belt; as they were designed to run form a single AA battery powering them should be easy.

Yes I've seen more open and DIY solutions. I was just hoping for a commercial version that I could buy off the shelf.

I find it quite expensive too. I guess the price is fair if you include the research they did, plus the prototyping, manufacturing and company overhead.

I've tried a prototype version of this belt once (for a couple of hours), it worked surprisingly well. But integrating the vibrations as a "new sense" into your brain (so that your subconscious can use it) takes at least some days.

In 1996 I was prototyping a belt with piezo transducers spaced around it with each doubling as emitters and receivers for a crude ultrasonic distance, and then each would vibrate at a higher frequency the closer the object came.

Nicknamed 'batbelt' in-house.

I dropped it after looking into insurance requirements and ADA compliance.

Did you get it to work? What was the sensation like?

Yes, and I used a Basic Stamp, and replaced it with a PIC chip later on. I had several normal issues: ignoring very close objects like your arms swinging when you walk; short postives; timing issues with getting the whole array working, since I was trying to triple use piezo transducers as emitter/receiver/vibrator. Other than those issues, it was not too refined due to the limitations of the transducers, speed of the 8-bit chips I was using, and you had to wear it within proximity of your body, but then not cover it up.

Before I ditched the idea, I was going to put the vibrator belt as a second belt worn under the clothing, but there was no Bluetooth or WiFi or easily accessible way of going wireless between the belts in a practical fashion.

I had my neighbor, blind man who had a guide dog, trial it. He was a great guy, and helped me quite a bit to 'see' past my vision biases. Unfortunately, he passed away. He used to work for the IRS, and we would joke about me getting my deductions past him! He did like what I had accomplished, but many blind people do not have the money for something that is not subsidized, and then you get caught up in the compliance insurance issues. Not sure if that has changed much nowadays.

It was a great maker project before the 'maker' movement. Too bad there was no startup, or crowd-funding web sites back then!

People adjust to reversed image in a matter of weeks https://www.theguardian.com/education/2012/nov/12/improbable...

It's not so far from that

It's no surprise to me that one would adapt over time. The article you link didn't mention any long-term study. I'd be curious to know how long it would take to replace one's vision to that of vertically or horizontally mirrored vision, in full. It's one thing to walk or drive a motorcycle, it would be another thing to handle situations requiring sub-second reflexes.

Once you've managed to fully adapt to the new mapping, is it just as jarring to revert and go back to normal, or have you learned how to handle both?

Say you rotated for an entire year - or longer - on a weekly schedule between normal vision, vertically mirrored vision, and horizontally mirrored vision. Do you gain full training in all three to the point that you could transition between normal/vertical/horizontal at any time without any disorientation or readjustment period? Or can the brain really only handle one mapping at a time? This would be the ultimate question to answer.

I suspect the results would be similar to how we handle multiple languages. Your primary vision (ie: the one you learned first, or the one you have used exclusively for a decade or more) would remain the easiest to work with, while acquired alternatives may approach the "fluency" of the primary, while not being perfect.

I assume it's very analogous to Destin's "backwards bicycle" - https://www.youtube.com/watch?v=MFzDaBzBlL0

Later on in the video, he mentions that having gotten used to the backwards bike, a regular bike was nearly impossible to ride until he adjusted to it again.

A human brain is very plastic. There is an app that can turn a picture from the phone's camera into sound real-time, and after a few minutes of training you can start to "see" the picture from listening to that sound.

Fascinating stuff.

What app is that, Artlav?

Not sure by now - it was shown to me last summer.

The thing basically scanned a line across the image, and produced sound based on the brightness. So, with your eyes closed you could quite easily imagine a basic shape of the image from the sound.

Maybe someone would recognize the description.

This thread inspired me to look this stuff up. I think you're thinking of vOICe? https://www.seeingwithsound.com/

Yes that's the one! https://youtu.be/XuosPzluCRg

It's a process called Sensory Substitution[0]. It's technically the perception of that sense.

I understand that the rest of your brain 'trains' this pathway to occur. So because you know that there is a sound source and you can see it move in space, and you 'feel' this on your tongue as well, you are capable of correlating the two senses together and associating them with each other.

[0] https://en.wikipedia.org/wiki/Sensory_substitution

Reply on news.ycombinator.com