Hacker News Reader

Comment by sp332 | Hacker News Reader

I'd like to find a way to start with an embedding and have the computer generate some text that corresponds, at least approximately. There are tools that do that for images, right? Like Stable Diffusion, you can put an image in, get an embedding, then do gradient descent in latent space to find a new embedding, then generate a new image from that.

View on HN

This is basically how RNN encoder/decoder architectures worked. The encoder encoded the input as a single vector and the decoder would decode this into text (eg. for machine translation) [1]. However, fixed-length vectors generally required too much 'compression' to represent variable-length text, so people started adding attention mechanisms so that the decoder could also attend to the input text. And the seminal Transformer paper by Vaswani and others showed that you only need an attention mechanism and you could ditch the RNN (hence the title 'Attention is all you need') and here we are.

So, this has been possible already for quite a long time.

[1] https://arxiv.org/pdf/1409.3215.pdf

To be sure, the seq2seq style already allowed multi length embeddings before transformers were a thing. RNN vs Transformer is entirely an implementation choice; you can build a seq2seq model with any combination of transformer, RNN, and conventional layers.

Each of these layer types have different computational costs for training and inference, and encode different inductive biases, which may be more or less appropriate to a given problem.

For some models, perhaps you can use get_output_embeddings() like in this example: https://stackoverflow.com/a/65267429

You can get this fairly trivially with word and sentence embeddings just by running the inverse (huggingface models have this as a builtin). For llama, the same is possible, but the matrix transpose is your responsibility :)

Ok, I got annoyed at oobabooga (around when it first came out) and have been messing with llama.cpp since then. I can't tell if this feature request is the same thing we're talking about here? https://github.com/ggerganov/llama.cpp/issues/1552 I guess if I want more features, I should move on to something that has more features lol.

Looking at that issue, I wonder how is it that everyone seemed to not understand the poster's question?

The feature itself is something I wanted to play with too, as it's kind of an obvious thing to want. I mean, these models execute a pipeline:

[text] -> [tokens] -> {[embeddings] -> [inference] -> [embeddings]} -> [tokens] -> [text]

Where the part in { ... } may or may not be implemented as a single step (i.e. all three parts interleaved).

Now, apparently all the magic[0] of transformer models sits in the latent space and is invoked by the { ... } bit. We also know for sure that you can make the pipeline look like this:

[text] -> [tokens] -> {[embeddings] -> [inference]} -> [embeddings]

So with the two things in mind, it's kind of obvious you'd also want a pipe that looks like:

[embeddings] -> [inference] -> [embeddings] (and optionally -> [tokens] -> [text])

for the sole purpose of messing around and exploring the latent space itself.

I'm very much not up to date with the whole space, so I might be missing something, but I'd thought that poking around the latent space would be getting a lot more attention than it seems to be getting.

(EDIT: replaced < ... > with { ... } for readability.)

[0] Not the "how do transformers tick" details, but the "how the hell are they this good" / "GPT-4 is uncanny valley" / "could this thing be actually thinking?" kind of magic.

In every model I’ve worked with, all of those steps are EXPLICIT. You take input, then tokenize, then embed, feed to model, then convert from embeddings to tokens, then back to words. There’s actually a lot of work done in embedding space! Chained execution, neural graph reasoning (see https://arxiv.org/pdf/2303.14617.pdf), even basic IR tasks like clustering and retrieval.

There’s been no real effort to especially expose it because that is what you get by default. Even OpenAI has an “embed” endpoint. So you’re not going to see a huge push for it the same way you won’t see a push for reasoning about websites “in the HTML”. :)

I only see a way to get embeddings out. How do I put embeddings into an OpenAI API?

im not sure i follow if you want embeddings for any word/sentence you send it to the embeddingendpoint and save the vectors coming back?

The point is that we want to shove arbitrary embeddings vector into an LLM and inspect what comes out. This is different from feeding tokens as input. The kind of thing I'd like to do, for example, is to:

- Get embeddings for e.g. "blue" and "red", or "the sky is blue" and "galaxy redshift";

- Average them, resulting a vector that's bound to not be expressible with tokens alone;

- Input that to the same model I got the embeddings from, and see what comes out.

If by "embeddingendpoint" you mean OpenAI, they provide that for a specialized model, and (AFAIK) you can only get embeddings out (for the purpose of comparing various vectors yourself). They have no API endpoint for an LLM that can take those embeddings as input.

Run the resultant embeddings through local beamsearch to get tokens, then run the detokenizer to get back words.

You can't and you can only get embeddings from a tiny gpt-3 model.

The most basic of embedding (that I can think of) is one where the number of dimensions corresponds to the number of unique characters in your lexicon. If there is a number in one of the components of your embedding that is greater than "0", then you know what characters they are. This embedding does not encode the order of the characters, though. They are just a "bag of characters". If you were to then also encode the order of the characters in, say, yet another embedding, you could use those two embeddings to recreate the original word.

Combine the two embeddings into a new vector space and BAM, you've invented "embedding2word".

Gpt (and many others) just add these embeddings together in the model, so you could do that and have one vector that encodes both things together

That seems suboptimal, for the same reason LLMs are trained on tokens, and not characters. Tokens seem like a much better "unit of meaning" than characters.

Curiously, this also applies to humans: we learn words first, spelling later; we almost always think in terms of words, subwords and phrases, and we're really fast at it - while anything to do with spelling seems to demand much more focus, and some degree of conscious hand-holding.

As for encoding sequence, I'm curious how that happens and need to find relevant papers, but I imagine there might be some "n-gram dimensions" in the vector, as in one value for "I'm a first token in a sequence", one value for "I'm a second token in a sequence", etc., which would encode the "occurs before"/"occurs after" relationships using few dimensions, leaving the rest for more interesting relationships.

Reply on news.ycombinator.com