Bot-controlled piano emulating spoken voice (2009) [video] | Hacker News Reader

Bot-controlled piano emulating spoken voice (2009) [video]

youtube.com

112 pointsprawn24 comments

Not mechanical, so much lower resolution... but you might have recently seen the History of Japan video being linked around... the guy who did that, Bill Wurtz also did this:

https://www.youtube.com/watch?v=YBI-nZTUgf8

There's an entire subreddit of this stuff as I guess it's really good practice for playing by ear:

https://www.reddit.com/r/zappafied

It includes this awesome Palin one:

https://www.youtube.com/watch?v=YBI-nZTUgf8

P.S. In case you missed it this is the History of Japan one again: https://www.youtube.com/watch?v=Mh5LY4Mz15o If you watch it again, you can hear him using that technique of harmonizing with his own spoken voice in the background music of that video.

That reminds me of the metal band Spastic Ink playing voices, sounds and music from Bambi.

original: https://youtu.be/X5SWc_k_Hmk?t=16m59s

with Bambi: https://www.youtube.com/watch?v=UhP4lg28fs0

Something awesome happened to me. I've known the album well but I didn't know the track was from Bambi. It was a weird track. Some time later I decided to watch Bambi because I was into animation classics and it was awesome all the time thinking "where have I heard this before?" Thank you for making me remember this awesome experience I've had.

This technique is similar to one Steve Reich's incorporated in a lot of his music, most notably Different Trains (https://www.youtube.com/watch?v=LsRpfbTI1EA)

I'm 5 minutes into /r/zappafied and it's already my new favorite thing. Thank you!

Ha... there's a guy named Chassol who does the same live. Lovely to find more.

See also this version of "All I want for Christmas is you" converted to MIDI.

It's a really great illusion I think.

https://www.youtube.com/watch?v=psIdan9GSRM

It's like with those "Play X backwards to hear the Devil" type videos. You understand it perfectly with subtitles, but close your eyes and it's gibberish.

I remember when this first came out there was some discussion (possibly here on HN?) that even the sound in the video was post-processed by overlaying some of the original voice. Although I can't seem to find it now, there's a recording of the piano without this editing, and it's far less comprehensible. In other words, they're cheating a little.

This article talks about that effect (with a similar example):

http://www.theatlantic.com/technology/archive/2014/06/sounds...

great link thanks! it's like the McGurk Effect https://youtu.be/G-lN8vWm3m0, and why I time Vocal Text to appear just before you hear it https://youtu.be/b-jrV7LyN7E

( Strangely enough, that's also how I 'hear' most politicians.... )

I always though this was amazing, and a great disappointment that there weren't more videos and documentation about the project. It clearly took a lot of effort, and it's a shame that there's nothing more than a news report to show for it.

The possibilities are intriguing!

This reminded me of the speech synthesis using pipes which William Gibson described in Neuromancer, I think.

I'd like to see this tried with a church organ. Or a set of flutes.

Very interesting !

I guess it's possible to use a MIDI synthesizer instead of the physical piano to achieve the same result ?

All that's needed is the conversion from the voice spectrum to notes/chords...

The added advantage is that it would be possible to use the pitch wheel to achieve intermediate notes for higher fidelity..

The output can then be processed again and it's pitch changed and so we can create music from synthesized speech. Does this make any sense :) ?

https://www.youtube.com/watch?v=ecm5PKTiVhw

Thank you for bringing Paul Lansky to my attention! A radically new soundscape (for me at least). Any more recommendations?

I don't have a specific recommendation, but Lansky sounds a lot like the Tape Musician's of the 50's. Said musicians would slice and glue together countless snippets of sound recordings of real objects or spoken voice to create their compositions. Today, the same can be accomplished digitally. Here is a recent example (though a little pop-music like): https://www.youtube.com/watch?v=pAwR6w2TgxY

It's just quantization at that point.. Aka pixie sound filter (also known as bit breaker.)

I'm really curious about the build of the bot. Are the "fingers" that press the keys pneumatic? I'm about to start an unrelated project but it does need fast responding "pressers", which might be classified as actuators. What are the benefits of using air powered vs electric? Does a bot like this have a huge can of compressed air or need to be hooked up to a pressurized line?

Seeing those banks of enormous capacitors, I'd say simple solenoids.

Pneumatic is one approach. Simple and inexpensive but with a lack of control over actuation speed.

It would be expensive, but linear actuators would give incredible speed and precise control over position and velocity.

Thanks for the info, much appreciated for a hardware noob like me.

Consider the low resolution of this piano speech. Pay attention at how many notes are used to create that low resolution speech, then imagine how many notes/frequencies are really used in natural speech.

As an amateur singer it's amazing and frightening how big is the range to sing out of tune.

Assuming it's not all homegrown, can anyone determine what software he was using besides MPlayer?

great. now my nightmares have a voice. :)

I do love the fact that someone had a non-trivial odd-ball idea and followed through.

Reply on news.ycombinator.com