Digitizing speech on a piano | Hacker News Reader

Digitizing speech on a piano

blog.makezine.com

79 pointsiamelgringo30 comments

This is awesome. I think the title is misleading though; it's about reproducing digitized speech. This converts the already digitally represented sound to actual analog sound, audible by humans. The output device is a by cleverly (and mechanically) played classical piano.

The resulting voice sounds like a bad special effect, but is really cool since it isn't. :)

However, if you don't look at the subtitles it's quite hard to recognize the words. Showing the subtitles of what you're supposed to hear is called 'Prompting', an effect which is also used by backward-speech advocates.

Right, but....

Your dog reads books!? - Yea, but only fiction.

Your piano can talk!? - Yea, but not very well.

http://www.youtube.com/watch?v=ZA1NoOOoaNw

That's hilarious... and illustrates the awesome power of prompting.

This is steampunk, man. Even more pimp than that demo with synthesized speech on a Commodore VIC-20.

Note to self: create better post titles. :) http://news.ycombinator.com/item?id=862590

If you read up a bit on how mp3 compression and GSM work then you'll find that in essence they rebuild the sound from a very limited number of frequencies.

Very nice find, and extremely impressive the speed with which the keys are manipulated.

I wonder if it is possible to have a human learn the key sequences of the dominant frequency bands and make a piano speak a word or two.

In neuromancer there is a head in villa straylight that speaks with organ pipes iirc.

> I wonder if it is possible to have a human learn the key sequences of the dominant frequency bands and make a piano speak a word or two.

I was wondering that too. Imagine if a composer in pre-digital history had figured out something like that and as a final "chord" to a piece had let the piano speak a word.

Literally awesome.

You mean something like that? - http://www.youtube.com/watch?v=g9D-kUEp03c

It's fascinating to listen to but I wonder about extending it to other languages. Does American English lend itself well to this sort of thing or are other languages and accents reproducible. I imagine that languages with much harsher stops like German or Russian would be more difficult and those that are easily broken down, like Japanese, would be even easier than English.

This seems like it might have some similarities to sine wave speech synthesis. Of course, its far more impressive with a real piano.

http://www.scholarpedia.org/article/Sine-wave_speech

Too bad it didn't mention how many hours of time went into designing and programming that system. It must have been an incredible amount of work.

Ah, the power of a least-squares fit. :)

There doesn't seem be any fitting involved. I think he just took the speech spectrogram and used it as a piano score. Thus the acoustic output of the piano is not meant to approximate the acoustics of talking. We hear speech nonetheless because there's enough of a speech-like signal for our auditory functions to hold on to.

The human auditory system does a remarkable job of filling in missing data. Sometimes it's amazing.

For example we can:

Hear a song we recognize, playing softly in the background, in a crowded noisy room

Talk and understand conversations over the telephone even with it's amazingly small bandwidth

Understand speech with something like 70% signal loss

Hear the cry of a baby in the middle of a battlefield over the sound of guns

and on and on and on

I find it particularly interesting that words that end in hard consonants (like T or D) are represented by very high-pitched keys to create the percussive sound (especially noticeable at the end of the video). This is too cool.

Very very amazing!

On another subject, I want some kind of 'piano player' with pistons like this piano, and you attach it to a non-MIDI instrument and it plays the keys via MIDI data, without having to retrofit it electronically.

Seeing as it is on MAKEzine I am upset there are not plans to build your own... Seriously that would be one hell of a weekend(s) project. Writing the software/midi stuff sounds fun too!

Very interesting but the damn reporter talked too much. I wanted to hear more of the piano.

IMHO it needs more cowbell.

And they need to explore the studio space more.

This is the coolest thing I think I have ever seen (or heard for that matter).

I've always wondered whether this could be done with a piano... thanks!

Kulturzeit is probably the best program on german TV..

more examples of this technique here:

http://ablinger.mur.at/docu11.html#qu3

My piano already does this. I can talk into a mic and it converts it into notes. It just doesn't mechanically play the keys.

So Sparky's Magic Piano could really have happened after all :)

wav2midi?

Anyone?

Reply on news.ycombinator.com