ACL 2023 Tutorial: Retrieval-Based Language Models and Applications

acl2023-retrieval-lm.github.io

27 points·TalktoCrystal··7 comments

While on the subject of ACL 2023 tutorials, here are two I went to:

Complex Reasoning in Natural Language https://wenting-zhao.github.io/complex-reasoning-tutorial/

Generating Text from Language Models https://rycolab.io/classes/acl-2023-tutorial/

the second one in particular was very well taught. I didn't catch the Retrieval based talk, since it conflicted with the text generation one.

This is useful

Were using retrieval-based system for Q&A for a complex product. Feeding in user guides, release notes even transcribed some tutorial videos. Its quick to create a 'wow' style demo but putting into production is challenge with hallucinations and testing/benchmarks hard to do given the stochastic nature of the output each time your run with same input (temp>0). Curious on pointers if anyone is doing something similar.

I'm curious what LLM your using. I would think some of the open source ones could be trained to use the facts given with much more accuracy.

Openai Gpt4. retrieval works best given the data source although debating if we should explore tuning as a next step

If anyone has any good pointers/references/benchmarks on how to evaluate such systems that would be really helpful

yes, the market is asking for this.

Reply on news.ycombinator.com