Pink Trombone | Hacker News Reader

Pink Trombone

dood.al

575 pointserrozero75 comments

In a similar vein: http://www.adultswim.com/etcetera/choir/

The most interesting thing about this one is the chord progressions it generates.

that is incredibly satisfying to play with

If you enjoyed that, the developer, David Li has a lot of other great projects: http://david.li/

The same team did a squishy Morty head for Adultswim. http://www.adultswim.com/etcetera/elastic-man/. Chris Heinrichs (Dolphin Club Audio formerly of Enzien Audio, https://dolphinclub.website/) does the procedural audio models.

You’re right, the progressions are beautiful! Do you know how they are generated?

Very impressive indeed. I dug around and found his tweet: "uses a neural network I trained on choral pieces to generate the harmonization for your melody"

[0] https://twitter.com/daviddotli/status/1075068713936830464

Man, shoot. I compose a bit myself and am really into vocal harmony and I was hoping that he'd hard-coded some theory rules.

I did notice that it seems to vary with the past 3 notes you've moved to. If you continue repeating the same pattern of jumps among the scale, you can replicate the chords the voices slide to.

> I compose a bit myself and am really into vocal harmony and I was hoping that he'd hard-coded some theory rules.

Would you find this useful? Is this not already available somewhere? Sounds like a fun project!

Yeah, I think I'd find it useful- I'd love to be able to ape a few of the tricks that the four voices use to slide to pleasing chords. The music theory "rules" are certainly available, but only in the same sense that the rules of calculus are available somewhere.

Either way I agree, definitely a fun project.

If you're interested in chord generation, I've used Cthulu[0] by Xfer records to experiment with chord progressions quickly.

[0] https://xferrecords.com/products/cthulhu

Cubase Chord Pads are also pretty sweet!

Praise Him.

In a historical vein, the absolutely magnificent Voder, from the late 1930s:

https://youtu.be/TsdOej_nC1M?t=16

https://en.wikipedia.org/wiki/Voder

I found it difficult to tell how well it's actually speaking because the announcer is priming the audience for every utterance. Very cool though!

After your comment, I muted the audio when the announcer told the audience what the computer would say. I couldn't get it straight away, but when they emphasised different words in the sentence I could usually tell what was being said "Greetings everybody!".

I didn't expect that to be so beautiful.

A little while back I used this combined with a physics simulator to make a toy where you throw polygons in the air and they scream: https://ohgodwhathaveidone.stackblitz.io/

Code's here if anyone wants to play: https://stackblitz.com/edit/ohgodwhathaveidone – I did a fairly medium job of abstracting the synthesis engine away from the UI, but it might be a decent starting point if you're looking to make other Trombone-based web silliness.

This reminds me of the Sprechmaschine [1] ("speaking machine") built at the end of the 18th century by Wolfgang von Kempelen (the guy who build the original mechanical turk [2]). Here is a YouTube video showing it in action (for example, the machine says "Mama" around 1:14): https://www.youtube.com/watch?v=k_YUB_S6Gpo

[1] https://de.wikipedia.org/wiki/Wolfgang_von_Kempelen#Die_Spre...

[2] https://en.wikipedia.org/wiki/The_Turk

also, the 1939 voder comes to mind :

https://en.wikipedia.org/wiki/Voder

https://www.youtube.com/watch?v=0rAyrmm7vv0

Literate C version: https://pbat.ch/proj/voc/

Literate depot: https://github.com/PaulBatchelor/voc

Actually compiled source is there: https://github.com/PaulBatchelor/Soundpipe/blob/master/modul...

Lest we forget that speech synthesis is not just for grotesque but amusing semi-real vocal synths like this, here's a BBC Radio 4 history of speech synthesis as an assistive technology - Klatt's Last Tapes, by Stephen Hawking's daughter, Lucy:

https://www.youtube.com/watch?v=097K1uMIPyQ

Man this brings back memories. We used this program for my linguistics homework in college. In 1996. Although I think it was an app then not a web page.

This would be useful to demonstrate the difference between p/f and l/r for those brought up without those distinctions.

I'd also (as an English speaker) like to see/hear Dutch g and Xhosan clicks.

yes, I think it's time for this to be connected to text input for seeing how any word is pronounced.

specifically: https://twitter.com/shaunlebron/status/989192507828432896

Perfect.

Would it be possible to use Reinforcement Learning + Speech Recognition to turn this thing into a real voice synthesizer?

I wonder if this sort of model would lend itself well to program induction similar to this paper: https://web.mit.edu/cocosci/Papers/Science-2015-Lake-1332-8.... Having a mouth seems like it would enforce a strong inductive bias.

No need. This thing is already a voice synthesizer. This is how modern synthesizers work, more or less: by generating a sine wave and then modifying it in the same way as the vocal tract does.

I meant "text to speech engine" then I guess.

Yeah, I know what you meant. You use a tool like the CMU pronunciation dictionary[1] to turn words into phonemes, and then you use a model similar to the pink trombone to turn the phoneme string into sound, including the transitions between different phones (which, it turns out, actually matter more than the phones themselves for making it understandable). This is how TTS works.

1 http://www.speech.cs.cmu.edu/cgi-bin/cmudict

Not a sine wave. The vocal tract is subtractive; the larynx has to generate a waveform with lots of harmonics some of which the vocal tract can then remove.

Reminds me of Xiph's Speex/CELP model of speech as a mix of noise and frequency to achieve high compression, requiring as little as 2.15 kilobits (275 bytes) per second. It sounds perceptibly similar to the original recording, even though the difference between the input and output sampled data may be high:

https://www.speex.org/docs/manual/speex-manual/node9.html

Bitrate comparison:

https://www.speex.org/comparison/

Samples:

https://www.speex.org/samples/

Maybe higher compression can be achieved with better prediction, aka machine learning.

I've actually been looking for the opposite of this (i.e. sound in, mouth representation out) for a while. Does anyone know of such a thing?

Yes! Oculus makes an SDK for this. You can use it in Unity 3D, Unreal, or directly in a native app. https://developer.oculus.com/documentation/audiosdk/latest/c...

Thanks! That's like 80% of the way there. It looks to be missing a lot of state internal to the mouth (understandable given that it's targeting avatar lipsyncing), and appears to discretize the values somewhat, making it less useful for linguistics practice. But I bet the underlying technology could be adapted easily.

Wow! I was able to successfully recreate all sorts of letters and sounds just by imagining how my own mouth works, and then manipulating the different components on the pink trombone in the same way. I'm impressed!

Pondering what makes this sound "male".

Vocal chord.

You can make it sound female by dragging the voicebox control to the right and down. It requires both higher pitch, and less power in the odd harmonics.

I agree it gets closer, but as is with much of speech synthesis, it still to me ends up sounding like "a male talking in a high-pitched voice" and not "a female".

In addition, this would imply that only males can talk in a low register, which is patently false. Low register female voices are fairly common.

It's a bit of a subtle distinction. A lot of voice acting on cartoon boys is done by women (classic example: Bart Simpson)

So they can make it sound more masculine but still high pitched

I have actually used this very tool back in the day to help learn how to speak in a male or female voice. One of the five things I do is manipulation of my tongue to change the cavity of my mouth to make the space bigger (more masculine) or smaller (more feminine) which this tool demonstrates very well.

Edit: to be clear I used to sound male 24/7 and now I sound female 24/7 Rather than thinking you are speaking male or female it helps if you think you are playing a musical instrument with a number of controls that you control (with your mind whahahaha). Then it is just about learning what each control does and how to play them so you get the result you want.

Your voice is muscle memory so while at the start I had to actively "play" a female voice that is no longer the case and now if I ever want to "play" a male voice I have to actively think about how I am going to speak each word to make it male.

This is also exactly how male countertenor singers produce a female-sounding voice, by making the vocal cavity smaller to adjust the formants upward. If you don’t do that, it just sounds like a male head voice or falsetto.

It has to do with formants. It's briefly explained in this video, which also demonstrates changing a female voice to male: https://youtu.be/nPAINeIGxMc.

Sudden sound warning.

Sudden disconcerting sound warning!

This is amazing. I can get it to make almost any speech sound, but one I can't get is [s], because the model lacks teeth!

disable "always voice", and click a bit below hard palate, slightly to the right towards the lip (below the at in palate), so there will only be a small gap for the air to go through

You're right. It sounds to me like [s] with lip rounding, where the teeth don't contribute as much acoustically, but it does indeed sound like an [s].

Were you able to get "m" and "n"?

Yes, just click in the nasal cavity above 'lip' and 'hard palate', respectively, to get [m] and [n].

The model also lacks mouth width/lip shape, which is crucial for differentiating between Swedish vowels (for example i vs y, which have most of the mouth the same except for y has the lips protruding and i is more of a smile).

I feel like I am simulating orgasmic responses!

This was also ported to C++ inside a modular synth:

https://www.youtube.com/watch?v=PDn7ygnJUfI

https://www.youtube.com/watch?v=3jcqKnIa8T4

https://www.youtube.com/watch?v=bo5ZEgBEapk

https://github.com/giuliomoro/pink-trombone

What someone needs to do is put sensors in people's mouths, record them saying known phrases, then stick the sensor data+phrases into some AI and see if we can't get that Trombone talkin'!

Previous post in Apr 2017: https://news.ycombinator.com/item?id=14135658

The shape of the tongue control reminds me a lot of the rhombus in https://en.wikipedia.org/wiki/International_Phonetic_Alphabe...

... which is because the latter was patterned after the shape of the mouth.

I feel embarrassed playing with this

You should have seen the looks from my coworkers when my computer started shouting AAAAAHHHHHHH

Thats nice and everything but until I have a hardware version where I can just switch a button on and do this and that, I'm not really going to be truly satisfied. Please hardware'ify.

how do i use this to answer the phone?

Is the author aware of the other meaning of "pink trombone"?[1]

[1] https://www.urbandictionary.com/define.php?term=pink%20tromb...

I was scared to click on the parent link at work because I KNEW this term had to have a second "urban dictionary" meaning...

How anglocentric

The words "pink" and "trombone" are english words, so there's that...

I'm fairly sure there is no other meaning than the one to which you refer, so its perhaps more likely that you are missing the risqué point being made by assuming the prude levels are higher than you might think. A lot of the folks who make these kinds of hacks, are perfectly fine with the obscure, obscene, perverse nature of their naming of things ... Those of the anthropological inclination may decide that, in fact, an obscene name for something like this is a requisite.

Like how in jargon.txt they used 69 as an example of a big number:

"69 adj. Large quantity. Usage: Exclusive to MIT-AI. "Go away, I have 69 things to do to DDT before worrying about fixing the bug in the phase of the moon output routine..." (Note: Actually, any number less than 100 but large enough to have no obvious magic properties will be recognized as a "large number". There is no denying that "69" is the local favorite. I don't know whether its origins are related to the obscene interpretation, but I do know that 69 decimal = 105 octal, and 69 hexadecimal = 105 decimal, which is a nice property. - GLS)"

For some reason, dirty names are common practice in audio tools. Some examples: "Rectal Anarchy" is another vocal synth for Buzz, grANALizer (emphasis not mine) is a popular granular audio effect, and even in the commercial world, it gets more subtle but it still exists: Image Line sells a plugin called "Gross Beat", which is a bilingual en/fr dirty joke.

Reply on news.ycombinator.com