Hacker News Reader

Comment by twiss | Hacker News Reader

Halfway through the article, he shows a picture with mouths for a-z, sh and th: https://cdn-images-1.medium.com/max/1600/0*gwbWfYm-iQIPEXXP%... There are others if you search images for "lip sync animation". It would be interesting to see if the result is understandable if you simply replace the emojis with those images.

View on HN

That image does not map to letters, it maps to the phonetic alphabet. English has different phonetics for words when compared to their written counterparts. For example "money" needs to be converted to "mahnee" before being converted to emoji counterpart. Otherwise it is gibberish.

'mah-' is US English, not English :P It's 'mah' in one country.

p.s. Seems every day on here I'm reminding people from the US that their country is not the world, their version of English is spoken and spelt in just one country, etc etc etc. Maybe I should give up doing that.

The most classic maybe was a linguist (yes, amazing) who on her blog was gloating that her variety of US English was particularly rich, in having almost every vowel sound in English. I looked at her list.. it didn't include the 'o' sound in 'pot'..

Either you don't understand that he means the pheonetic alphabet, or you say money in a way that is foreign to almost any person in the UK as well as the US. Do you say mooney or mohney? If not I think your phoneme almost certainly must be mah.

Hi, sorry about the delay. I don't know what you mean, 'he means the phonetic alphabet'. I say the first syllable to rhyme with 'bun'. That's surely fairly standard (non-US) English. It's mysterious to me that that's not one of your alternatives.

British English has at least two pronunciations of money that I know of, /mʌni/ and /mʊni/

The former is the most common way for Americans to pronounce it, and matches the mouth shape of ah, and the latter, while not common in the US, also matches the ah mouth shape. I really don't know how yesenadam is pronouncing it such that his mouth doesn't take the ah shape.

Ohhh. I'm sorry everyone. Yes, you're right of course. It didn't occur to me until I tried it that the bu- in bun is bahhh. Goes back to drawing board

"Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize."

..which would be, what?

The fact that phoneme letters should have been used instead of alphabet letters in order to decide what mouth shape the icons should have.

Lipreading is different. For example the «sh» sound is made by slightly pursing the lips but not all speakers do this. In German it gives a very clear visual picture.

I would only purse my lips if I were performing a shush. In words like ‘hash’ I’d just close my mouth slightly & leave lips parted and wide from the ‘ha’ sound.

I agree that the mouth shapes are garbage in this demo. A shame because that is the most interesting & important part of animating speech.

This demo is done with my nearly 0 knowledge in lips reading, the effect I was searching was to make it more realistic, but with the limited amount of usable emojis I cannot have it very realistic, just better than random. At the end of the article I propose some gotchas to improve it: use better images, use a better map.

I'm happy that this article has triggered some people curiosity and can maybe evolve in something more advanced

That's why I am bad at lipreading in English. People have mouth shapes all over the place. I'm sure I could learn lipreading a friend I meet often well, however this doesn't help lipreading a stranger. When traveling in the US I always have a notebook ready.

Reply on news.ycombinator.com