Hacker News Reader

Comment by Bromskloss | Hacker News Reader

Yay, a sound-field camera! I've been wondering if such things are made. Should I take the existence of this project as an indication that they aren't?

The use case I imagine is finding what part of a machine is emitting a noise, much like how you would use a thermal camera for finding what part of it is running hot.

View on HN

Not exactly unheard of, but quite rare. I found a company that made large arrays like that, but they seem to operate in near field, measuring only the sound volume.

Imagine a wall of microphones next to a truck - you can tell which part of the wheel makes the sound. That sort of stuff.

As for an actual "camera" that turns sound into an image - i've never heard of one.

> As for an actual "camera" that turns sound into an image - i've never heard of one.

I have. His name is Daniel Kish [1, 2].

Apparently his visual cortex has learned to process sound rather then light, so he literally sees, based on the sounds he is hearing.

[1] http://www.abc.net.au/catalyst/stories/4441831.htm

[2] https://en.wikipedia.org/wiki/Daniel_Kish

Unfortunately only he can appreciate his own experience.

He gives a pretty decent explanation of what he is seeing in the Catalyst transcript. It's fascinating, as from his description it's a low resolution (due to the longer wavelength) of what you or I would see. Presumably he also sees things such as colours, but based on audio reflectivity rather than optical reflectivity?

Your point about only him appreciating his own experience is interesting, as the same question can be asked of all of us [1]. In audio, maybe apples and oranges are the same colour and strawberries are different? It would be an interesting application of a sonic camera, to render the image so that equivalent audio colours are consistently mapped to the same light colours.

[1] https://www.quora.com/Visual-Perception-Is-everyones-experie...

Isn't this exactly what sonar does?

SOund Navigation And Ranging: https://en.m.wikipedia.org/wiki/Sonar

Not exactly. A sonar maps a surface using sound, while my camera looks at the sound itself.

Think laser rangefinder on a scanner platform vs a camera.

Have you seen light field photography [1]? It would seem that your array is quite similar in concept. I'd imagine that getting images similar to those produced by various SONAR imaging technologies is a similar sort of deconvolution problem. If you're looking for textbooks or scientific papers to review, I think the field of tomography is also ripe with methods to explore here.

I'd challenge the notion however that this isn't 'just' a passive SONAR device. Keeping the camera analogy, active SONAR use spread spectrum pings, much like the white light of a camera's flash. Your camera lacks a flash function, and its lens is maybe not yet as in focus as it could be.

If you did want to add a flash, you're likely a $0.02 piezo device and maybe $20 in other various parts (and heaps of numerical processing on a PC) away from having the ability to produce a nice 3D reconstruction of the physical scene.

If you get direct data capture going, please do post some raw data online. I'm sure others would have heaps of fun playing around with it.

1: http://graphics.stanford.edu/projects/lightfield/

Hm, why not - here is the raw data (after PDM decoding) for the 16x16 video from the article: http://orbides.org/etc/snd_16x16_raw.tar.gz (4 Mb)

One file per cell, 16bit integer samples. Stored row by row, 128 slices per frame.

Not sure where to post it, however - here it would be buried.

There ARE sonars that deliver an image.

http://blueprintsubsea.com/starfish/images/sonar/452/sf452_s...

That's just one example - look for side scan sonars for the earliest stuff (https://en.wikipedia.org/wiki/Side-scan_sonar) or sonar imaging for the latest stuff (http://www.tritech.co.uk/product/gemini-720i-300m-multibeam-...)

Such systems are ubiquitous on commercial fishing vessels.

https://en.wikipedia.org/wiki/Fishfinder

No. SONAR sends out an ultrasonic ping and times the response of its echo to get a distance.

This project is similar in that it involves visualization from audio, but apart from that, it's quite difference. This is the ability to see a sound. To derive the positions of arbitrary sounds in a viewing plane, and render it like you would a picture or video.

Yes. Passive sonar only listens to the noise. Passive sonar systems are often arrays too. So the analogy holds quite well from a distance ;)

https://en.wikipedia.org/wiki/Sonar#Passive_sonar

He's still driving the scene - illuminating it, if you would.

No.

Most sonar is active (sends out pings) and has only one input (one microphone)

It builds pictures by directing the ping and measuring time until the response.

This is fundamentally different than an array of microphones passively listening to, and creating a fine grained picture of some sound source.

> Most sonar is active (sends out pings) and has only one input (one microphone)

While I believe the US Navy doesn't prefer the term "Passive Sonar", submarines and some surface ships are equipped with hydrophone arrays (either on the hull, or towed arrays, or both) as well as active sonar; this isn't an uncommon thing. (I was under the impression that the active sonar didn't use a single hydrophone, but instead used the same hydrophone array used for passive detection.)

Yes it is. Despite all the other replies asserting SONAR is active, passive SONAR systems are very much a thing.

For my undergraduate project years ago I built a microphone array that could be used to locate noisy objects. I demonstrated it by tracking aircraft taking off at a local airport.

Whilst I didn't make 'acoustic photos' with it, I did plot the flight track of the aircraft in real time, and could have a decent crack at identifying it based on frequency content.

Sonar sends out sound and then listens for the echoes though, rather than just take in sound from other sources.

Some types of SONAR send out sound, other types (e.g. [0]) passively listen for sounds and use beamforming to localise the source of the noise.

[0] https://en.wikipedia.org/wiki/Towed_array_sonar

Doesn't the army use something like this in the field to determine enemy sniper locations when they are taking pot shots?

https://en.wikipedia.org/wiki/Gunfire_locator

Congrats, beautiful history and nice hacking. I hope you get a really big bag of money one of these days so you can keep doing this kind of things!!! kudos!

I commented it elsewhere in this thread, but I a company called Sorama has been doing sound camera stuff for quite a while. Afaicr it's based on the phd dissertation of their CEO Rick Scholte so that name might yield some articles you might like.

Cool stuff!

There is a Swiss startup called Distran (http://www.distran.ch/) which develops ultrasound cameras. They also have a spherical version which looks really cool and allows 360deg sight, unfortunately I cannot find anything about that one on the website anymore.

A finnish one too: http://www.nlacoustics.com/ Can someone with more expertise explain the differences between the many companies in this field?

We all have a bit of a different way of approaching the problem and our target market differs somewhat. You can use this technology for a lot of different things, not just the obvious use cases. We're still looking at a lot of applications, so the clearer product differentiation is something that will form as time goes by.

- Kai @ nlacoustics.com

Squarehead makes exactly this product: http://www.sqhead.com

It's used my military & events for drone tracking by sound, and concert venues for seeing where sound leaks out to the surrounding neighborhood. (Among other uses)

Edit: Their example screenshot is even showing an engine block overlaid with acoustic output of the components: http://static1.1.sqspcdn.com/static/f/1431009/18268254/13373...

No, not common, but as the OP noted the evolution of MEMs based microphones might make them a lot more common. As the author notes, they are more like holographic cameras than what many people think of as "regular" cameras. By converting the imaging into the FFT domain the resolution gets better "everywhere" the more sensors you add. (If you have ever cut up a hologram into smaller and smaller pieces you can see this in action, as the full image is still there, just lower resolution). It would also be interesting to illuminate a non-active source with sound beam. By using two, slightly out of phase, ultrasonic sources he might be able to reveal depth detail of the surface being imaged.

Well as it happens there are special sonar cameras out there capable of providing real-time video images[0][1].

The use case here is being able for a diver or ROV to see in murky water

[0]http://www.soundmetrics.com/image-gallery [1]http://www.apl.washington.edu/programs/z_DIDSON/Pages/LIMIS....

Not quite the same thing though. These systems are active. They have sound being produces most likely at a very specific frequency and then listened for so that depth information can be gathered. This instead images the sound that's already out there in the light. It's the difference between lidar and a camera

Nope, theirs is the sound-camera, detector array and sound-lens, but with an acoustic floodlight. I think they do use ranging though, so the 2D image is "sound camera" in one raster dimension, and "sonar range" in the other.

Sounds to me more like the difference between using the flash or not.

Sort of, except the "camera" can't make any use of "light" generated by anything other than the flash.

http://visisonics.com/apps/#nsi

I'm unaffiliated, but saw one of their demos.

That's a very sensible use case. I was thinking more like, super-trippy live music visuals.

http://www.gizmag.com/seesv-s205-sound-camera-locates-noises...

Yours is sensible too.

Visisonics had a device that did something like that.

Search: acoustic camera. They sell phase-array microphones for mapping noise sources. Rather than a plane array, theirs is a sphere.

Reply on news.ycombinator.com