Also, I'm really linking to research groups, so take these names as starting points and look at their students and other professors working with them.
First off, the Blizzard challenge is a major hub of activity. [4] Festival is an important piece of software [5]. Interspeech is an important conference [6] (take a look at the speech synthesis track and the organizers for that).
Alan Black [0] @ CMU is kinda a giant.
Keiichi Tokuda [3] @ Nagoya also a giant.
Simon King [1] @ Edinburgh, just had a paper linked to on HN a few days ago and does important work.
Mark Gales [2] @ Cambridge does work here, too.
[0] https://www.cs.cmu.edu/~awb/ [1] http://www.cstr.ed.ac.uk/ssi/people/simonk.html [2] http://mi.eng.cam.ac.uk/~mjfg/ [3] http://www.sp.nitech.ac.jp/~tokuda/ [4] http://festvox.org/blizzard/ [5] http://www.cstr.ed.ac.uk/projects/festival/ [6] http://interspeech2015.org/wp-content/uploads/direct/INTERSP...