Hacker News Reader

Comment by forgingahead | Hacker News Reader

* Don't use a single embedding per content item, use multiple to increase retrieval quality

Can you share some specific examples of what you mean by this? How would you process specific info types (eg: news article, or web page, or product catalogue data) this way, and how would you handle retrieval that makes the quality "better"?

*Edit: Thanks for all replies so far - yes I am aware about splitting or chunking the data, but interested in a good write-up of techniques and pros/cons of each with examples. Eg: Chunking sentences vs. paragraphs, providing context around the embedding result, asking GPT to generate questions to chunks and embedding that instead, combining interaction data (eg: purchases or clicks after search queries) with actual content data before embedding, embedding attributes around data, and so on.

View on HN

There are a couple of things that can help. As has been pointed out by other commenters - chunking up content is very useful.

The other often neglected approach is to use an LLM to derive new content and use the embedding of this as well.

E.g. ask the LLM “give me a list of questions that can be answered by the following passage”

You then use embeddings of the generated questions instead of embeddings of the original content.

another approach is HyDE, where you ask the LLM to come up with a plausible (but likely wrong) answer, and use the embedding of the wrong answer to find the appropriate chunk, pretty clever

Presumably, they're referring to chunking up the data into discrete semantic units—smaller vectorizable subsections (e.g., paragraphs) more precisely capturing different parts of the data.

I'm curious about more sophisticated answers to this question, but the obvious approach would be to split the article or web page into sentences and do an embedding per sentence.

When I was playing around with search via embeddings (as a test I was using Vampire the Masquerade V5 sourcebooks, and asking rules questions), I got the best results -- in terms of correct answers -- by using sentence embeddings. I'd search the query against the sentence embeddings, and then retrieve more context surrounding the winning sentence(s). That context would be passed to the LLM.

It wasn't perfect, though. I'm tempted to try the avenue of having an LLM generate questions for each passage and then use those embeddings, but it sounds a bit expensive to set up given the length of the books.

Sentence embeddings + retrieving context makes sense.

By any chance, have you done any work with indexing code using embeddings? I'd like to do something similar there, but there's no obvious notion of "sentence", especially across languages.

Probably the closest analogue is just lines of code, but breaking lines on newlines might break an expression in the middle removing meaning from both halves.

I was planning on trying indexing overlapping groups of lines but haven't had time yet.

Reply on news.ycombinator.com