Having done some of this myself, I’m curious your results on fine tuning vs embeddings. I’ve found the latter much more performant, but perhaps I’m thinking about fine tuning wrong.
I also did a little work in letting it scan through my notebooks (they are OneNote and you can access and search via a Python API) via keyword search because it can point directly to something I've written in the past, and not just rely on model weights.