Visualize Latent Spaces
github.com
github.com
Second, disclaimer: I am not now and might never be a serious algebraic and/or differential geometer. Just a fan at the moment.
I've been calling the useful transformations in LLM latent manifolds things like "substantially affine", and I think that's probably true enough of the current crop.
I don't think this about `{V, I}-JEPA` (about which there's a lot of information and I plan to look into it a lot more) or Sora (about which there is less information but is still impressive AF). One imagines that `V-JEPA` and Sora have some deep parallels/symmetries.
Either way, I'll wager that serious Riemannian geometry is rapidly on it's way to table stakes. We have extreme high-dimension spaces that result from backprop and gradient descent, some combination of smooth/continuous/differentiable/compact seem pretty likely to fall out? Along with interesting curvature tensors and parallel transport for moving around in them? And TDA for figuring it out numerically/computationally?
I'd love if an expert chimed in, I'm trying to describe an intuition with a fluency that involves pointing and gesturing.
Continuous geometric concepts don't play super well with the way we like to decompose model outputs into discrete entities (classes, words, visual properties). We can, e.g. find variables in celebrity face GAN latent spaces that seem related to face orientation, or hair color, sort of, over some variable range and under some input conditions, but that doesn't really translate cleanly into any typical mathematical characterizations, geometric or otherwise, where you'd be looking for some property to hold everywhere or at least have an atlas of connected local approximations to simple characterizations.
Instead, we get high-dimensional messes of spaces, and network gradients during training don't exhibit clean or easy to understand dynamics except in the simplest toy cases.
To paraphrase a more serious "math for ML" prof I've chatted with at times -- "doing math" classically involves being able to find a description with only a few free parameters for a complex phenomenon that may superficially appear to have many/infinite free parameters. It's possible that for large ML models trained on natural data, such a reduction just doesn't exist, you can't break the contributions of millions or billions of parameters down into a low-dimensional approximation. He was/is skeptical of us attaining deep mathematical insight into their operation, but he could always be wrong. I'd certainly love to see cool novel insights come out of mathematics that give clarity to what's been going on these past 15 years.
This is obviously speculation/intuition, but it's not terribly surprising to me at least that operating in e.g. pixel-space or a straightforward lifted latent manifold (modern diffusers basically) wouldn't have apparent structure under the fancy t-SNE type things that seem to be the heaviest artillery brought to the party (at least in the open). In pixel space, you get 6-17 fingers on 1-3 hands.
The `france - paris + uk === london` thing is real, and it's not surprising because there aren't typically much in the way of nonlinearities in `word2vec`/`fasttext`/`glove` type stuff. But this substantially survives all the leaky relus or whatever in LLMs. They're pretty clearly interpolating in a way that you could get close to with a composition of affine transforms.
JEPA (and maybe Sora if..., fuck it) seems a dramatic shift in forcing joint loss into a much higher-level space/manifold with (to me at least) shockingly semantic properties. I mean look at the I-JEPA reconstructions from pre-trained lifted space with some dinky diffuser/VAE-thing eating the hyperplane:
https://ai.meta.com/blog/yann-lecun-ai-model-i-jepa/
That's not pixel space, and you've got a lot of freedom to make it smoother, I suspect no one says "L1 regularization" anymore, but there's some modern version of that, we know how to do this.
AFAIU (and again, I welcome expert correction) TDA at least and really a lot of modern geometry is about "scruffy intrinsic / smooth embedded" or vice versa, and "scruffy at this scale but smooth if you set the focus right".
Edit, obviously I don't know my zonotope from my tropical hypersurface, I do however like the pretty pictures ;)
Here's another good piece of how topology might help work out why some models do better than others; https://journals.plos.org/ploscompbiol/article?id=10.1371/jo...
"By computing geometric descriptors of DNNs and performing large-scale model comparisons, we discovered a geometric phenomenon that has been overlooked in previous work: DNN models of high-level visual cortex benefit from high-dimensional latent representations. This finding runs counter to the view that both DNNs and neural systems benefit by compressing representations down to low-dimensional subspaces [20–39, 78]. "
When it comes to the math underneath an LLM, https://medium.com/autonomous-agents/part-8-mathematical-exp... is about the most accessible explanation I have found so far.
I don't know if this line of research is going anywhere, but I thought it was interesting in terms of actual geometry being applied to ANNs.
- by visually linking similar entities, vs just a scatter plot of values floating around or binning, you see a lot of the substructure and non-linkages these tools otherwise mislead you on
- you get a full interactive suite for on-the-fly visual encodings, drilling, reclustering, etc for wheb you start investigating it
- automatic support for heterogenous data. Ex: for user analysis, combine profile + name embeddings with numeric, date, Boolean, etc features, so you can work with the real dataset, not just a couple columns
For video/images, there are other tools I'd recommend because they add another layer of nuance. But for this kind of text, tabular, etc data, we've come to be opinionated and built in a lot here over the last few years, including with cool work by our collaborators at Nvidia (rapids)
I wrote a little tool last year for myself that I called "hyperspace" haha- it allows me to do a similar "inspection" of model activity and output across a series of visualizations.
If I have a new kind of data, not text, not image, so there is no existing embedding, how do I create one?
Any good articles/talks on this?
Usually, you train a neural network to solve some kind of task with your data. The most common task is probably classification, for example, "Is the animal shown in this image a dog or a cat?" or "Does this text sound happy or sad?".
Once your network is trained, you discard its last layer, which was responsible for classification, and use the output of the second-to-last layer as your embedding vector.
This works because the first few layers of the network have already transformed the data into a generally useful representation, which gets turned into specific classes by the last layer, or can be used as an embedding vector instead.