Hacker News Reader

As an alternative, it's not a bad idea to first convert each document to markdown via a thinking or agentic LLM. Embedded figures can even be embedded as readable tables or Latex or Mermaid. Do record the name and parameters of the model that performs the conversion. You can then query the markdown using any model with an input token cost that is exactly equal to the encoding of the markdown. For multiple queries you can also use input caching.

View on HN

Yup, tried all variations of that. There's the advantage that you can use a lower hallucination OCR specific model for the pre-processing, at least for clean text. But for something hard like handwritten forms, applying VLM with context is less error prone than preprocessing to text.

Also - and this is bizarre - the token cost of doing that is higher, not lower, at least in Gemini world, and by a large margin. That's very counterintuitive, but a page encoded as image tokens can be smaller than same page as text, and is not meaningfully lossy on documents that are just typed text because the models are well trained on those.

Reply on news.ycombinator.com