Good question. Do you need OCR too, or just file recognition? Models like Granite are pretty good for simple OCR in a tiny footprint, but not punchy enough for serious work.
Only that the model can search existing pdf:s, docs, etc and I can then query it. Seems like the obvious local use case. I can’t upload gigabytes of data to ChatGPT, even if I wanted.