This is something Google has been working a lot on [1] and Baidu also recently posted about their results too [2]. We're definitely pretty close to passing the human detectable level.
[1] https://deepmind.com/blog/wavenet-generative-model-raw-audio...
[2] http://research.baidu.com/deep-voice-production-quality-text...
The human mind seems to be better at this than most creators credit it for.
Speech models usually use triphones, which turns out to be a huge amount of audio. This is particularly impressive because of how little data they need.
Google used their own datasets, which are most likely massive.
To me, at least, the voices I heard in Lyrebird's demo actually sounded more 'real' than Microsoft Sam for example.