Harnessing the Universal Geometry of Embeddings
arxiv.org
arxiv.org
1. An MIT professor who works on similar geometry alignment problems didn't want to work on this with me because he was certain we would need at least a little bit of paired data
2. A vector database startup founder who told me about his plan to randomly rotate embeddings to guarantee user security (and ignored me when I said it might not be a good idea)
The practical takeaway is something that many people already understood, which is that embeddings are not encrypted, even if you don't have access to the model that produced them.
As one example, in the Cursor security policy (https://www.cursor.com/security#codebase-indexing) they state:
> Embedding reversal: academic work has shown that reversing embeddings is possible in some cases. Current attacks rely on having access to the model [...]
This is no longer the case. Since all embedding models are learning ~the same thing, we can decode any embedding vectors, given we have at least a few thousand of them.
Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication.
Assuming that is true. If you can detect when you are trying to put a square peg into a round hole, does this mean you have the ability to remove square holes from a system?
> Does this extend to being able to analytically determine which concepts are encodable in one embedding but not another? An embedding from a deft tiny stories LLM presumably cannot encode concepts about RNA replication.
Yeah, this is a great point. We're mostly building off of this prior work on the Platonic Representation Hypothesis (https://arxiv.org/abs/2405.07987). I think our findings go-so-far as to apply to large-enough models that are well-enough trained on The Internet. So, text and images. Maybe audio, too, if the audio is scraped from the Internet.
So I don't think your tinystories example qualifies for the PRH, since it's not enough data and it's not representative of the whole Internet. And RNA data is (I would guess) something very different altogether.
> Assuming that is true. If you can detect when you are trying to put a square peg into a round hole, does this mean you have the ability to remove square holes from a system?
Not sure I follow this part.
My thought there was that you'd be comparing tinystories to a model that trained on the entire internet. The RNA related information would be a subset of the second representation that has no comparable encoding in the tinystories space. Can you detect that? If both models have to be of sufficient scale to work the question becomes "what is the scale, is it sliding or a threshold? "
>> Assuming that is true. If you can detect when you are trying to put a square peg into a round hole, does this mean you have the ability to remove square holes from a system?
>Not sure I follow this part.
Perhaps the metaphor doesn't work so well. If you can detect if something is encodable in one embedding model but not another. Can you then leverage that detection ability in order to modify an embedding model so that it cannot represent an idea.
1. possibly high loss where the models don't have compatible embedding concepts 2. given a sufficient "sample" of vectors from each space, projecting them to the same backbone would show clusters where they have mismatched concepts
It's not obvious to me how you'd use either of those to tweak the vector space of one to not represent some concept, though.
But if you just wanted to make an embedding that is unable to represent some concept, presumably you could already do that by training disjoin "unrepresentable concepts" to a single point.
> We introduce the FIRST method for translating text embeddings from one vector space to another without any paired data
(emphasis mine)
Nope. I'm not gonna go a literature search for you right now and find the references, but this is certainly not the first attempt to do unsupervised alignment of embeddings, text or otherwise. People were doing this back in ~2016.
For example, global rotation that does not change embedded vector x embedded vector dot-product and changes query vector x embedded dot-product in an equivariant way.
EDIT: Perfect symmetries, for example, feature-less spheres, or the analogues of platonic solids would break this. If the embedded space has no geometric symmetries you would be in business.
Re-aligning, essentially would be akin to solving a graph-isomorphism problem.
Lie algebraic formulation would make it less generic than an arbitrary graph-isomorphism problem. Essentially reduce it to a high dimensional procrustes problem. Generic graph isomorphism can be quite a challenge.
https://en.m.wikipedia.org/wiki/Procrustes_analysis
EDIT: Sinkhorn balancing over a set of points (say a d-dimensional tetrahedron, essentially a simplex) furthest from each other might be a good first cut to try. You might have already done so, I haven't read your paper yet.
The GAN works way better than traditional OT methods though. I really don't know why, this is the part that feels like magic to me.
All the best.
I can totally understand the professors point, little bit of alignment data ought significantly increase the chance of success. Otherwise it will have to rely on these small deviations from symmetry to anchor the orientation.
On a per document basis, would the universal embeddings of the smaller models (less performant) cluster around the better model's universal embedding space, in a way suggestive that they are each targeting the 'true' embedding space, but with additional error/noise?
If so, can averaging the universal embeddings from a collection of smaller models effectively approximate the universal embedding space of the stronger model? Could you then use your "averaged universal embeddings" as a target to train a new embedding model?
I used to work on what your paper calls "unsupervised transport", that is machine translation between two languages without alignment data. You note that this field has existed since ~2016 and you provide a number of references, but you only dedicate ~4 lines of text to this branch of research. There's no comparison about why your technique is different to this prior work or why the prior algorithms can't be applied to the output of modern LLMs.
Naively, I would expect off-the-shelf embedding alignment algorithms (like <https://github.com/artetxem/vecmap> and <https://github.com/facebookresearch/fastText/tree/main/align...>, neither of which are cited or compared against) to work quite well on this problem. So I'm curious if they don't or why they don't.
I can imagine there is lots of room for improvements around implicit regularization in the algorithms. Specifically, these algorithms were designed with word2vec output in mind (typically 300 dimensional vectors with 200000 observations), but your problem has higher dimensional vectors with fewer observations and so would likely require different hyperparameter tuning. IIRC, there's no explicit regularization in these methods, but hyperparameters like stepsize/stepcount can implicitly add L2 regularization, which you probably need for your application.
---
PS.
I *strongly dislike* your name of vec2vec. You aren't the first/only algorithm for taking vectors as input and getting vectors as output, and you have no right to claim such a general title.
---
PPS.
I believe there is a minor typo with footnote 1. The note is "Our code is available on GitHub." but it is attached to the sentence "In practice, it is unrealistic to expect that such a database be available."
We tested all of the methods in the Python Optimal Transport package (https://pythonot.github.io/) and reported the max in most of our tables. So some of this is covered. A lot of these methods also require a seed dictionary, which we don't have in our case. That said, you're welcome to take any number of these tools and plug them into our codebase; the results would definitely be interesting, although we can expect the adversarial methods still work best, as they do in the problem settings you mention.
As for the name – the paper you recommend is called 'vecmap' which seems equally general, doesn't it? Google shows me there are others who have developed their own 'vec2vec'. There is a lot of repetition in AI these days, so collisions happen.
Sorry if I'm being obtuse, but I don't see any mention of the POT package in your paper or of what specific algorithms you used from it to compare against. My best guess is that you used the linear map similar to the example at <https://pythonot.github.io/auto_examples/domain-adaptation/p...>. The methods I mentioned are also linear, but contain a number of additional tricks that result in much better performance than a standard L2 loss, and so I would expect those methods to outperform your OT baseline.
> As for the name – the paper you recommend is called 'vecmap' which seems equally general, doesn't it? Google shows me there are others who have developed their own 'vec2vec'. There is a lot of repetition in AI these days, so collisions happen.
But both of those papers are about generic vector alignment, so the generality of the name makes sense. Your contribution here seems specifically about the LLM use case, and so a name that implies the LLM use case would be preferable.
I do agree though that in general naming is hard and I don't have a better name to suggest. I also agree that there's lots of related papers, and you can't cite/discuss them all reasonably.
And I don't mean to be overly critical... the application to LLMs is definitely cool. I wouldn't have read the paper and written up my critiques if I didn't overall like it :)
Imagine having more than a passing understanding of philosophy, and then reading much of any major computer science papers. By this "No right to claim" logic, I'd have you all on trial.
The goal here is harder: given an embedding of an unknown text in one space, generate a vector in another space that's close to the embedding of the same text -- but, unlike in the word alignment problem, the texts are not known in advance.
Neither unsupervised transport, nor optimal alignment can solve this problem. Their input sets must be embeddings of the same texts. The input sets here are embeddings of different texts.
FWIW, this is all explained in the paper, including even the abstract. The comparisons with optimal assignment explicitly note that this is an idealized pseudo-baseline, and in reality OA cannot used for embedding translation (as opposed to matching, alignment, correspondence, etc.)
Intuitively I assume this would work even better.
Do you think that could be sufficient to translate the Voynich manuscript?
Say I want to build an app with embedding/vector search. Currently, my embeddings are generated by model A, that is not open source. Later, I find a better embedding model B, and my new data will be using this model B. Since A and B are two different vector spaces, how can I migrate A to B, or how can I make vector search work without migrating A to B?
Can your research solve this problem? Also，if all embedding models are the same, is there a point of upgrading the model at all? some must be better trained than others?
I still do not understand exactly where D1L comes from in LGan(D1L, T(A1(u)). Is D1L simply A1(u)?
I also find that mixing notation in figure 2 and 3 makes it tricky.
Would have loved to have more insights from the results in the tables.
And more results from inversion, on more than Enron dataset. Since that is one end goals, even if reusing another method.
Thank you for the paper, very interesting!
Given the demonstrated risk of information leakage from embeddings, have you explored any methods for hardening, obfuscating, or 'watermarking' embedding spaces to resist universal translation and inversion?
No, we haven't tried anything like that. There's definitely a need for it. People are using embeddings all over the place, not to mention all of the other representations people pass around (kv caches, model weights, etc.).
One consideration is that's likely going to be a tradeoff between embedding usefulness and invertability. So if we watermark our embedding space somehow, or apply some other 'defense' to make inversion difficult, we will probably sacrifice some quality. It's not clear yet how much that would be.
And there's obviously always ArXiv. Maybe we should make a blog or something, but the updates really don't come that often.
The idea that a data set is "different" merely because its some subset of this maximal corpa is a difference without a distinction. What isnt being proposed is, say, that training just on all the works of scifi fiction lead to a zero-info translatable embedding space projectable into all the works of horror, and the like (or say that english-scifi can be bridged to japanese-scifi by way of a english-japanese-horror-corpus).
The very objective of creating LLMs with useful capabilities entials an extremely similar dataset starting point. We do not have so many petabytes of training data here that there is any meaningful sense in which OpenAI uses "only this discrete subspace" and perplextiy, "yet another". All useful LLMs sample roughly randomly across the maximal corpus that we have to hand.
Thus this hype around there being a platonic form of how word tokens ought be arranged seems wholly unevidenced. Reality has a "natural arrangement" -- this does not show that our highly lossy encoding of it in english has anything like a unique or natural correspondence. It has a circumstantial correspondance in "all recorded electronic texts" which are the basis for training all generally useful LLMs.
Could we also use this to read the Voynich manuscript, by converting Voynichese into embeddings and embeddings into English text? Perhaps, though I worry the manuscript is too short for that.