I’d love if Firefox would feed the text content of each website I visit locally and would allow me to RAG search this database. So often do I want to re-visit a website I visited weeks earlier but can’t find it again.
Here is the list of technological problems:
1. When is a page ready to be indexed? Many websites are dynamic.
2. How to find the relevant content? (To avoid indexing noise)
3. How to keep an acceptable performance? Computing embeddings on each page is enough to transform a laptop into a small helicopter with its fans. (I used 384 as the embedding dimension. Below, too imprecise; above, too compute-heavy).
4. How to chunk a page? It is not enough to split the content into sentences. You must add context to them.
5. How to rank the results of a search? PageRank is not applicable here.
Also, don’t discount plain old BM25 and fastText. For many queries, keyword or bag-of-words based search works just as well as fancy 1536 dim vectors.
You can also do things like tokenize your text using the tokenizer that GPT-4 uses (via tiktoken for instance) and then index those tokens instead of words in BM25.
https://ollama.com models also works really well on most modern hardware
And as to sidestepping inference, I can totally do that. But I think it's so much better to be able to ask the LLM a question, run a vector similarity search to pull relevant content, and then have the LLM summarize this all in a way that answers my question.
From what I can see, they both have the ability to archive / FTS your bookmarks.
But in terms of API access, historious only allows WRITE access (ugh), where at least pinboard allows read/write.
What else am I missing?