It turns out that accurately predicting text using a model that's a couple orders of magnitude smaller than the inputs requires understanding what the words mean.