Hacker News Reader

You missed the point - you had to train BERT or anything similar to get useful results out of it.

Now all you need is to give it more context along with your query.

Exactly. And there’s a lot of situations where context change frequently and fine-tuning / re-training becomes impractical or even impossible. With a general classifier + context, you can change the context dynamically and get instant results. That capability opens up a whole lot of possibilities.

But aren't BERTs tiny compared to a LLM and can be trained cheaply with the help of an LLM?

Suppose you want it to make a decision based on ..say.. 300KBs of somewhat changing information per query.

There's no scenario where you are training BERT online to give you an answer.

Can any model give you a reasonably correct answer on that amount of data?

Yes

Reply on news.ycombinator.com