Hacker News Reader

Comment by DebtDeflation | Hacker News Reader

>knowledge base (files)

What is it actually doing with the files you upload? Is it just pasting the full text into the prompt? Or is it doing something RAG-like and dynamically retrieving some subset based on the query?

View on HN

This is undocumented (frustrating) but it looks like it's chunking them, running embeddings on the chunks and storing the results in a https://qdrant.tech/ vector database.

We know it's Qdrant because an error message leaked that detail: https://twitter.com/altryne/status/1721989500291989585

It only applies that mechanism to some file types though - PDFs and .md files for example.

Other file formats that you upload are stored and made available to Code Interpreter but are not embedded for vector search.

Interestingly, when I used an .md extension, GPT would write python code to try to pull parts out to answer queries (which worked miserably), but when I used .txt (for the same files), it seemed to put it in the vector store.

I've had good success putting code examples in a single txt file for our custom framework, and it seems to use that neatly for generating code. I'm surprised you've not had much success with them, I gave an assistant my wife's PhD thesis and while the API was working initially it seemed alright.

Does that error message really confirm qdrant for Chat? It's just failing to index a file called 'qdrant', and I don't see any further proof offered in that thread.

That's good enough evidence for me.

They're clearly running a vector store (you can get further hints at that by spying on the JSON using browser DevTools).

Qdrant is a very good vector store - it's powering all of the new Twitter features as of a few weeks ago.

Seems much more likely to me that they're using Qdrant than this is a weird error message coincidence.

I think they are running their RAG on Microsoft's Azure Cognitive Search. Satya mentioned something about it in the yesterday Microsoft Ignite keynote

Actually msft CEO mentioned in his presentation that OAI moved to Azure's vector search and AI search services for ChatGPT.

If the file is short, they put the full text into the prompt. If it's longer, they use some sort of RAG with qdrant, it appears to be top-1 with context expansion, but nobody's knows for sure how they're doing the chunking.

I've seen gpts with a SQLite file that is loaded by the code interpreter to provide structured data as needed

I've seen gpts playing games with code that was attached to the prompt generating the turn results

I've seen gpt using API for converting YouTube to text to provide summaries

There's a lot of things that can be done, even if it's not the most dev friendly experience

Reply on news.ycombinator.com