> Word vectors are awesome but you don’t need a neural network – and definitely don’t need deep learning – to find them

Word2vec is not deep learning (the skip-gram algorithm is basically a one matrix multiplication followed by softmax, there isn't even place for activation function, why is this deep learning?), and it is simple and efficient. And most of all, there is no overhead using word2vec, just a difference between pre-trained vectors or trained ones.

I don't understand what this article tries to say.