It's pretty unfair to give it character level tasks, when it's input is probably tokenized with subword units. I am already a bit surprised that it even knows which letters go to which words.