From LSA/SVD you get a V x K matrix as well - that's exactly what the factorisation is doing.

The following two papers also go into detail about the mathematical similarities between LSA and neural embeddings and achieving similar performance with both:

Levy, O. and Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. https://www.cs.bgu.ac.il/~yoavg/publications/nips2014pmi.pdf

Levy, O., Goldberg, Y., and Dagan, I. (2015). Improving distributional similarity with lessons learned from word embeddings. http://www.anthology.aclweb.org/Q/Q15/Q15-1016.pdf