Hacker News Reader

Comment by tarasglek | Hacker News Reader

This looks cool, but I'm confused as to how this is surfaced in your product, llama-8 is not present in your model list.

I thought maybe you offer hallucination detection, but I also don't see that. RAG evals also not visible

View on HN

Thanks for the q.

The LSR version of AutoFeedback is on our Pro tier. You can sign up for more info on that here: https://tally.so/r/w2XVQA

In the free trial of Log10.io you can use the ICL (self-improving) version of AutoFeedback for hallucination detection and RAG evals. In generalize, you can customize evals for a wide range of use cases using AutoFeedback: https://docs.log10.io/feedback/auto_feedback https://github.com/log10-io/log10-cookbook/tree/main https://arjunbansal.substack.com/p/hybrid-evaluation-scaling...

As for models, we support evaluating a wide range of models including Llama - in addition we support OpenAI, Anthropic, Gemini, Mistral, MosaicML, Together, and self-hosted models. We are also compatible with frameworks such as Langchain, Magentic and LiteLLM. For how to integrate different models please see here: https://github.com/log10-io/log10

Also wanted to address the confusion on the role of llama-3-8b and llama-3.1-8b. In the blog post we use these models as an example of an evaluator LLM. We select what is best for your custom eval under the hood. LSR is just one example of the research powering our custom evals

Reply on news.ycombinator.com