Hacker News Reader

This was a wonderfully accessible intro to word embeddings and how they work. Worth a read for anyone just starting out with NLP or just with a passing interest in it.

View on HN

It doesn't seem to cover contextual word embeddings though which are all the rage since BERT came in.

You're right, I too was a bit disappointed that the page didn't mention contextual embeddings since that has been the best performing and most robust approach in NLP for nearly four years now.

It's still a clear introduction to word vectors though and it is good to know where we came from.

Agreed, and appropriately timed! I just finished using one-hot encoding for a predictor on student practice. The results were better than traditional methods, but only marginal better. When discussing the results I specifically mentioned how the N didn't understand meaning or levels of difficulty between exercises. This helps add more context to my own understanding and will definitely guide my next study.

Reply on news.ycombinator.com