I think they're trying to say there is a good use case for everything. For instance, if you have a set of keyword-like IDs (normalized items like EntityID:47283 for example) for a container of these things, like documents, you can make a quick GLoVe or W2V representation for each entityID, and then the cosine similarity between an entity vector and itself is exactly 1, not 0.98 or something.

So in the scenario that these are products which have IDs, and the container is a document that mentions these products together, you can reduce it to a key-value map of documentID-[list of product IDs], and then by selecting the vectors that have cosine similarity of .95-0.9999, you can get the object similar to that product but not that product. Using contextualized vectors, in the best case would give you things similar as well as the query product, but for every single instance, not just the single abstract entity.

No, a tag like a hashtag.

Think Instagram or Twitter tags.