Would love to read a more in depth write up of this if you have the time !
I suspect the obsessive note-taker crowd on HN would appreciate it too.
I suspect the obsessive note-taker crowd on HN would appreciate it too.
The hybrid retriever piece has its own deep dive with the RRF math and an interactive fusion calculator: https://blakecrosley.com/blog/hybrid-retriever-obsidian
See what your coding agent thinks of it and let me know if you have ways to improve it.
You wrote:
>A search for “review configuration” matches every JSON file with a review key.
Its good point, not sure how to de-rank the keys or to encode the "commonness" of those words
For the remaining noise, I chunk the flattened key-paths separately from the values. The key-path goes into a metadata field that BM25 indexes but with lower weight. The value goes into the main content field. So a search for "review configuration" matches on the value side, not because "configuration" appeared as a JSON key in 500 files.
MiniLM-L6-v2 is solid. I went with Model2Vec (potion-base-8M) for the speed tradeoff. 50-500x faster on CPU, 89% of MiniLM quality on MTEB. For a microservice where you're embedding on every request, the latency difference matters more than the quality gap.