Hacker News Reader

Comment by stymaar | Hacker News Reader

> They recently bought a Claude subscription and began using Claude to do the initial read of the documents and output JSON they can import into their internal systems. The work still must be reviewed by an attorney - Claude is nowhere near making the kinds of judgments a lawyer would make about this content - but it has increased their throughput from 2-3 documents an hour to 8-10 documents an hour by killing the busy work.

At my company we're also in the process of deploying a system that does exactly that. And what's interesting is that you absolutely do not need a frontier model for that, a small VLM (vision-language model), with optionally a little bit of fine-tuning, gives you the same output quality at a fraction of the latency and cost.

View on HN

This is really where Gemini still excels. Honestly from my experience the others aren't as good.

Yes, we used Gemini before and it worked fine in terms of output quality.

But between 10 and 20 seconds to analyze a document with only a bunch of pages just isn't good UX.

And what are you using now?

So which one do you use now?

Which Gemini? 3.8 Flash is fast and very nearly as good as Pro.

I'm wondering if the bottom is going to fall out of frontier models when people start to realize this. Sure, as an experienced programmer you can tell the difference between a compact free model and the ones that are 100x bigger and cost billions to train, but 90% of busy work is dead simple: turn a PDF into json fields, or do the inverse and fill out a form. As the tools for this conversion become more widespread you can also imagine an economic shift toward efficient information brokers that make frontier models less necessary.

It's also not really clear if everyone is going to want a frontier model when the real implications sink in. Maybe we'll get sick of incomprehensible code optimizations and people wile tire of reading AI prose that feels ever-more-human. There might be a few use cases, but who is going to pay for this when providers start charging enough to be profitable.

I think this is the long-term reality of LLM tech and one of the objectives of engineers implementing solutions, finding the right fit for the job - the right model and cost to achieve high enough accuracy at the lowest price.

But that's phase two, phase one is finding localized problems to solve using LLMs and productize them. I'm reminded of cloud tech, where phase one was changing software to run in the cloud, and phase two was optimizing costs.

Yep this is one of the best use-cases for a fine-tuned SLM/SVLM.

Reply on news.ycombinator.com