Hacker News Reader

16k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full article realm and that is already incredibly useful.

Is there any open model with a similar context length? [I'm not talking about the dubious for long context fine-tuned LLaMA variants, I mean the real thing.]

View on HN

Try looking into vector databases to solve that problem.

You can chunk up a book and embed those partitions into a vector database. Then you can take a query and fuzzy match the most relevant documents in your vector database, then feed it back to open AI to resolve an answer.

It's brilliant. Postgres has an extension to support indexing the vectors, and there are some other open source and turnkey solutions in the market as well.

Brilliant, but not the same. I think both approaches have their place and are not mutually exclusive.

They are not the same, but your initial problem of throw a whole book at it and have OpenAI give you an answer is a demo that is solvable in 20 lines of langchain code when leveraging a vector DB.

Demo here: https://www.youtube.com/watch?v=h0DHDp1FbmQ (github code is linked within as well)

Not open, but with Anthropic the limit is 100k now https://news.ycombinator.com/item?id=35904773

Reply on news.ycombinator.com