Hacker News Reader

Comment by killerstorm | Hacker News Reader

Hmm, why was their starting point not something like BERT:

  * already known as SotA for text classification and similarity 
     back in 2023
  * natively multi-lingual

View on HN

People generally sleep when you start talking about fine-tuned BERT and CLIP, although they do a fairly decent job as long as you have good data and know what you're doing.

But no, they want to pay $0.1 per request to recognize if a photo has a person in it by asking a multimodal LLM deployed across 8x GPUs, for some reason, instead of just spending some hours with CLIP and run it effectively even on CPU.

Are you suggesting use the clip embedding for the text as a feature to train a standard Ml model on?

I think he is. I do things like that plenty.

I think they're suggesting doing that with BERT for text and CLIP for images. Which in my experience is indeed quite effective (and easy/fast).

There have been some developments in the image-of-text/other-than-photograph area though recently. From Meta (although they seem unsure of what exactly their AI division is called): https://arxiv.org/abs/2510.05014 and Qihoo360: https://arxiv.org/abs/2510.27350 for instance.

I’m no ML engineer and far from an LLM expert. Just reading the article though it seemed to me that leveraging an SQL database here was a bigger issue than using traditional ML on the data, rather than the LLM being a win specifically. Just finding anything that was better suited than string matching on a RDBMS to the type of inputs seems like the natural conclusion when the complaint in the article itself was literally about SQL.

> they do a fairly decent job as long as you have good data and know what you're doing.

This is the bottleneck in my experience. Going for the expensive per-request LLM gets something shipped now that you can wow the execs with. Setting up a whole process to gather and annotate data, train models, run evals, and iterate takes time. The execs who hired those expensive AI engineers want their results right now, not after a process of hiring more people to collect and annotate the data.

>... as long as you have good data and know what you're doing.

I think you've just identified, in a set-theoretic complementary manner, the TAM for GenAI.

What's TAM?

https://en.wikipedia.org/wiki/Total_addressable_market

I love using encoder models, and they are generally a better technology for this kind of application. But the price of GPU instances is too damn high.

I won’t lie that I’ve been unreasonably annoyed that I have to use a lot more compute than I need, for no other reason than an LLM API exists and it’s good enough in a relatively small throughput application.

Because Enterprise Development only moves forward based on hyped technologies.

Reply on news.ycombinator.com