Very interesting comments. Yes absolutely want to do comparisons to ELMo. It's a little tricky to do so on our datasets, since ELMo isn't really a complete method on its own, but more an addendum to existing methods. In the future we hope to do seq2seq and sequence labeling studies, and we can then ensure we pick datasets that the ELMo paper covered.

Using char tokens can definitely be helpful, as can sub-words. It's something we've been working on too, and hope to show results of this in the future.

I mainly disagree with your view of end-to-end training. In computer vision we pretty much gave up on trying to re-use hyper-columns without fine-tuning, because the fine-tuning just helps so much. It's really no trouble doing the fine-tuning - in fact the consistency of using a single model form across so many different datasets is really convenient and helpful for doing additional levels of transfer learning.

Thanks for the note about table 7 - it's actually an error (it should be 5.57, not 4.57; Sebastian is in the process of uploading a corrected version).