Hacker News Reader

Comment by regularjack | Hacker News Reader

Implementing search correctly would solve the problem too, without requiring a square kilometer of GPUs.

Right but implementing search correctly often means humans combing through the data and applying common sense to tag the products with their actual facets. So really you're just choosing between a square kilometer of office space for cheap labor or GPUs.

> you're just choosing between a square kilometer of office space for cheap labor or GPUs.

No, with LLMs you're getting both, because a. LLMs need the human-curated data to ingest and b. after training the model is going to get RLHF.

Also, a square kilometer of office space still uses much less electricity than a square kilometer of GPUs.

I'm guessing we're at or very near the point where LLMs could tag the data better and cheaper than humans.

If you have >100k items, “implementing search correctly” is an intractable problem if you don’t have a hundred person team of search relevance optimizers and taggers. Search is terrible until you gather tons of metadata about each item. Search is not nearly as simple as text matching within product titles. An LLM by comparison is peanuts. You could run it overnight and just have it build tags and metadata for all your items and it would probably be 75% accurate and a hell of a lot cheaper. Things can then be fine tuned by people later.

At Amazon scale that last x% is a multi-decade problem and still going. It’s intractable full stop.

Reply on news.ycombinator.com