I'll probably throw away the waveform. I'm just looking for ways to use the audio to pace the text output, and also to tie the speech and text together as much as possible, so there's never a gap between them for your focus to jump between.
How are you synchronizing to each word?
I wonder if Google would consider expanding their TTS a bit to provide syllable data and other rich breakdown information.
I like the waveform, btw.
I'm using Polly for the moment, and it provides word and viseme boundaries. Enough to animate face poses. I'm not taking full advantage of the visemes here: it's only affecting the saturation of the waveform color. Too subtle to notice, with the way things kind of flicker with this one.
I have other ideas I want to play with, but don't really have time, and I may just have to drop that line of thinking, because I plan to move to one of the wavenet implementations, which don't provide speech marks data.