If you want them to be meaningful without changing the model...

Couldn't you rotate the basis to minimize the distance between each basis vector and it's nearest neighbor?

One of the fascinating things about neural embedding such as these is that the individual component dimensions have NO "real" semantic meaning to us humans. It's better to think of them as single points in a higher dimensional space.

(of course, with a clever network design you could probably FORCE "meaning" onto some components)