Hacker News Reader

Comment by ilaksh | Hacker News Reader

November 2022.. might have been a pretty weak model or just out of date info.

This to me is a cautionary tale against deploying cheap small LLMs instead of using larger models. I think 7b models are very tempting to many business people who may value profits over just about anything else.

View on HN

I just don't understand why companies don't use something like Algolia on their KB articles with some careful manual keyword curation.

Imagine all the person had to do was type "bereavement" in a search bar and it instantly matched with the bereavement policy. What more does a person need?

Exactly this. I feel like LLMs are great for so many things and have a crap ton of real-world use cases. But for simple FAQs, there's so many better and cheaper solutions out there.

... Larger models aren't immune from hallucinations, nor is the rate of hallucination negligible enough to ignore with larger models.

This is a fundamental issue with the underlying technology, one that many companies in this space refuse to reckon with.

A lot of "this can be automated with AI!" startups are relying on the basic assumption that hallucinations can be tolerated - cases like this really narrow the field of use cases where that is true.

Hallucination or inaccuracy is dramatically worse with 7b models versus the largest ones.

Just wait till an aggressive lawyer finds out about this. Gonna be a lot of companies taken to the cleaner for hallucinations

There is no evidence they used a small LLM. There is no evidence that LLM size is the issue. Environmentally bad to use a large model when a smaller one could work just as fine.

There’s no evidence it was even an LLM at all.

Reply on news.ycombinator.com