Thank you for explaining it. I was aware of the problem of too short context length. I'd love to be able to pass an entire programming project with my prompt for example (or a book in your example).

I think I understand how it works now for many kinds of prompts where the information to be extracted is contained in one(or more) of the chunks of much bigger whole.

I'm not sure about prompts where it is required to "understand" entire input to answer properly. For example summarising a book. Although even with this vector search could perhaps help by looking for things not near "please provide a summary", but certain hand crafted values such as "important to the plot" etc.

I guess I need to do some experimenting with It. I found some open source alternatives to (not at all)OpenAI in form of "Sentence Transformers" to create embeddings.

However, what would be really neat is to have a large open source dataset of embeddings already created on some general purpose collection of texts, to try searches etc.