Microphone turns any surface into a multitouch interface | Hacker News Reader

Microphone turns any surface into a multitouch interface

extremetech.com

65 pointsmrsebastian18 comments

The title is grossly misleading. This setup detects "gestures" which is not the same as the multitouch interfaces we use with our smartphones. Bloggers threw "multitouch" on the title; not the creator [1].

Nothing in the original article indicates that the setup is able to detect the location of the touch, only the type of touch. A thump, a tap, a swipe, or a scratch. The various gestures are mapped to different sounds, creating a virtual drum set, if you will. The fact that this detection, mapping, and playback happens so instantaneously is amazing.

I don't mean to take away from the accomplishment. This is an extremely novel idea, and the product implications are numerous. It's just a shame that the tech press is distorting the product idea in a way that creates a false expectation.

http://www.brunozamborlin.com/mogees/

Agreed. Unfortunately, this seems to be pretty common in online tech journalism. My own experience: We published a paper on another touch sensing technique last year which got featured in a short article for Technology Review. The author interviewed both of us and wrote a very good article. Over the following days, a number of sites like Engadget picked up the story and apparently only used content from the TR article. IIRC not a single author reported additional details or even contacted us to confirm this story.

This preference of speed over accuracy leads to stuff like 'Wi-Fi Makes Trees Sick, Study Says' [1].

[1] http://www.pcworld.com/article/211219/wifi_makes_trees_sick_...

Hrm, if you watch the video, the tone definitely changes with the location of the hit -- as in, with the same radius from the microphone, a different note is produced depending on where a finger strikes the surface. Gestures are also included, as you say.

With regard to multitouch -- a) some gestures involve more than one point of contact, and b) there are some bits in the video where it looks like two fingers are striking a surface at the same time and making two tones/noises.

Depending upon the type of material, the tone will change when you strike it. I still think it's a mistake to say that this turns a surface in to a "multitouch" surface, because of the vast differences between this and what has commonly been referred to as multitouch.

Again, extremely cool technology, but you couldn't for example, attach a microphone to a wall and couple this with a projector to control the OS. That's the inference that gets drawn when you say "multitouch".

Also this:

> ...contains multiple microphones to create a stereo image of the sounds it hears

led me to believe the device determined the direction of the sound source by measuring phase shifts in different microphones. Now that apparently this device doesn't do that, may I ask why not?

I can't find any reference to multiple microphones or phase shift analysis in the source material. This paragraph from the source material, specifically, leads me to believe it's using the acoustics of the touch, not the location:

> Through gesture recognition techniques we detect different kind of fingers-touch and associate them with different sounds.

Notice the language "different kind of", rather than "the location of". I just don't see anything here that would indicate spacial information is gathered.

That's understood, I'm just asking why not doing phase shift analysis with multiple microhpones, as another idea.

You could probably do something like this, but accuracy is going to be a problem. Sound waves are not uniform events. If you could rely on precise clicks at precise frequencies, with minimal echo, you could get acceptable accuracy, but relying on finger thumps in varied substrate is going to be difficult. I think the reason you don't see more of this is that there are better solutions to the problem of establishing touch interfaces on suitable surfaces.

Yeah, multitouch is a wrong word here.

This is a very creative idea and execution. The next step would be to use two microphones sitting at a distance to form a triangulation to determine the 2D location of the touch. Then multitouch can be truly supported.

I can't work out if that would be possible or not.

It might be possible to plot the location of a single point in 2D, but how would you differentiate for multitouch? If you have 2 microphones you might be able to calculate the distance from each microphone of a touch by timing how long it takes for the sound of the tap to reach each microphone. However, as the speed of sound in wood appears to be ~4000 m/s (http://www.engineeringtoolbox.com/sound-speed-solids-d_713.h...) you'd need to be "listening" at 400kHz to get centimetre accuracy over a 1m square area.

I can't see how it would be possible to then differentiate between two points moving at the same time (i.e. a pinch gesture). I'm just assuming regular microphones are used as the article suggests - I'm not an acoustic engineer so hopefully I've missed something.

Interesting problem. Haven't thought of the sampling rate required. The location resolution probably will be poor for precise multitouch function. But even a single 2D location is enormously useful - turning any surface into a touch screen.

kabdib mentioned phrased array, which can be very helpful. It might solve the sampling rate issue since it's analog. I found this http://en.wikipedia.org/wiki/Beamforming. May be some of the techniques in radar/sonar can be borrowed here.

Use a phased array of mics. Classic technology (well, 1960s).

Probably much harder with random mic placement.

Probably much harder with random mic placement.

You could run an initial calibration on the system by having the user tap a large number of points within the touch area and noting the relative phase, amplitude, and frequency content received by each mic for each location.

More or less same technique, presented at UIST in October 2011:

http://chrisharrison.net/index.php/Research/TapSense

Contrary to the bloggers report, there is more info and source available:

http://ftm.ircam.fr/index.php/A_brief_introduction_to_FTM

http://imtr.ircam.fr/imtr/Gesture_Follower

Not exactly the same, but worth mentioning:

Acoustic tablets have a long technological history, as seen from this 1971 patent...

http://en.wikipedia.org/wiki/Acoustic_tablet

Why Is this site so wrong with iphone safary?( stuck page, difficult to scroll down, unable to go back to HN...). I always promise myself I'll not touch again this web with a 5' pole( and always forget :/)

Reply on news.ycombinator.com