False; try making something similar with a Markov model trained on just 800 words: the RNNs have somehow learned what it means for a word to be pronounceable, and rarely output words which are difficult to pronounce.