Hacker News Reader

Comment by giraffe_lady | Hacker News Reader

Everyone wants this to be it but over and over we discover that the bigger a model is the better it is at all tasks, even ones far outside the domain it was optimized for. IE claude fable is better at writing both code and prose than smaller code- and prose-specific models.

The way vision and language models converge into the same geometric space should be extremely alarming for the "you don't need global knowledge for local tasks" type dreams.

And to be clear I'm not saying that smaller models don't or can't work well, or that we shouldn't be heading in this direction. And it's not quite the case that broad knowledge is strictly necessary. But it never seems to be negative! And so far it is the best way we've found to do... everything. Small models are good to the extent they are like big models, not to the extent that they are small.

View on HN

On narrow domains, it is very common for small models to match or outperform larger ones at a fraction of the parameter count.

For example in language, this is called the “curse of multilinguality”. Small models that handle a single translation direction can easily outperform big ones that try to handle them all.

https://arxiv.org/pdf/2311.09205

In any case, for most tasks the question is not “how many tasks can this model kind of do well” but “given time/cost constraints, what is the maximum level of quality we can achieve”. And for that, small models are usually very competitive.

I think the context here is that small models run locally, not rented from a cloud.

Yes small models are and will be useful for lots of stuff for several reasons.

But the idea they’d be better than a bigger model is cope, you’re pretty much always better off running the biggest one you can bring to bear within your constraints.

This argument seems somewhat strange to me, and at odds with the recent model releases.

It feels like claiming that because a current generation laptop CPU is slower than a current generation HEDT CPU, we should ignore that the laptop CPU is faster and more efficient than the previous generation HEDT CPU.

Of course a huge system could do more, but these systems are constantly evolving and just in this month we've seen comparatively smaller models approach capabilities that were frontier AI less than a year ago. What you're saying would be true when we eventually find the limits of how much models can compress knowledge. Until then, all I think you're saying is that companies that are willing to burn more money can afford to ignore efficiency.

> claude fable is better at writing both code and prose than smaller code- and prose-specific models.

There's a million ways to define "better at writing prose", and it's only better at some of them. It's worse at many others.

> Everyone wants this to be it but over and over we discover that the bigger a model is the better it is at all tasks,

Try using a LLM model for RAG embeddings and get back to us on that.

Why do you think I haven't done this?

If you have, then it explains so much more.

Reply on news.ycombinator.com