Hacker News Reader

Comment by nateburke | Hacker News Reader

Fixing this is the promise of enterprise deployments of LLMs, correct? That you can take the list of initial result texts from a shitty elasticsearch implementation, pass it into an LLM with a prompt "you are a user looking for <query>, please use common sense to comb through these results and find a few good ones" and get better intramural search quality.

The only other enterprise application I am aware of is applying the homework cheating app to customer facing roles that require email/copy generation.

View on HN

Implementing search correctly would solve the problem too, without requiring a square kilometer of GPUs.

Right but implementing search correctly often means humans combing through the data and applying common sense to tag the products with their actual facets. So really you're just choosing between a square kilometer of office space for cheap labor or GPUs.

> you're just choosing between a square kilometer of office space for cheap labor or GPUs.

No, with LLMs you're getting both, because a. LLMs need the human-curated data to ingest and b. after training the model is going to get RLHF.

Also, a square kilometer of office space still uses much less electricity than a square kilometer of GPUs.

I'm guessing we're at or very near the point where LLMs could tag the data better and cheaper than humans.

If you have >100k items, “implementing search correctly” is an intractable problem if you don’t have a hundred person team of search relevance optimizers and taggers. Search is terrible until you gather tons of metadata about each item. Search is not nearly as simple as text matching within product titles. An LLM by comparison is peanuts. You could run it overnight and just have it build tags and metadata for all your items and it would probably be 75% accurate and a hell of a lot cheaper. Things can then be fine tuned by people later.

At Amazon scale that last x% is a multi-decade problem and still going. It’s intractable full stop.

Reply on news.ycombinator.com