As for specific errors, does rock capitalize when "the" is before it? (The Rock) The models learn from how users write. But when you think about it, it is actually fairly difficult to predict what words are going to be, you can test it with your friends. Have them create a sentence and feed you one word at a time. The better you know the person the better you'll do, but I'm guessing you still won't be that good.