Hacker News Reader

Tools for exploring the models are already being developed.

Language models that are not given concepts of part-of-speech, dependency trees, grammars, develop representations of their own and people have found ways to inspect them.

View on HN

Could you please provide pointers to relevant papers? I'm very interested.

Here's a paper on how BERT (a large Transformer model trained using self-supervised learning) implicitly learns the traditional NLP pipeline: https://arxiv.org/abs/1905.05950

Reply on news.ycombinator.com