Hmm, typically the models where people use cosine similarity on embeddings have been deliberately trained such that the cosine similarity is meaningful. It looks like this paper is looking at examples where the models have not been deliberately trained for the cosine similarities, and hence in these situations it would indeed be unreasonable to assume cosine similarities to be a good idea.. (but that's kind of a given?)

For example, here's the loss from the CLIP paper [1], which ensures cosine similarities are meaningful:

    # joint multimodal embedding [n, d_e]
    I_e = l2_normalize(np.dot(I_f, W_i), axis=1)
    T_e = l2_normalize(np.dot(T_f, W_t), axis=1)
    # scaled pairwise cosine similarities [n, n]
    logits = np.dot(I_e, T_e.T) * np.exp(t)
    # symmetric loss function
    labels = np.arange(n)
    loss_i = cross_entropy_loss(logits, labels, axis=0)
    loss_t = cross_entropy_loss(logits, labels, axis=1)
    loss = (loss_i + loss_t)/2

And Sentence Transformers [2] using CosineSimilarityLoss:

    train_loss = losses.CosineSimilarityLoss(model)
    # Tune the model
    model.fit(train_objectives=[(train_dataloader, train_loss)], epochs=1, warmup_steps=100)

[1] https://arxiv.org/pdf/2103.00020.pdf

[2] https://www.sbert.net/docs/training/overview.html