Hacker News Reader

jjmachanon Show HN: Ragas – Open-source library for evaluating RAG pipelines

could you elaborate on what you mean by perf? cost we'll add soon

The emergence of LLMs has opened up a venue for tackling problems that were earlier thought impossible. The plethora of LLM-based applications is proof of this. But the one question still remains a mystery, how to effectively evaluate LLM-based applications?

We will try and solve that mystery through this article by understanding methods used to benchmark LLMs and discussing SOTA methods, available frameworks, and challenges in evaluating LLM-based applications.

jjmachanon Show HN: Complete guide to reward modeling for RLHF (with code)

This post consists of two parts. The first part explains the reward modeling process along with the gist of various important research that led to the evolution of reward modeling as we see it today. The second part is a step-by-step Python implementation and explanation for training a reward model.

jjmachanon The Past, Present and Future of LlamaIndex

This is our first interview series and I couldn’t have asked for a better guest other than Jerry Leu, the creator of LlamaIndex. LlamaIndex is a framework that helps you connect Large Language Models with your own data. This is something that opens up a wide range of possibilities that was not possible before. Things like building customer service bots to answer customer queries based on documentation, extracting insights from the huge amount of unstructured data in companies, paring through any source of information like books, videos, podcasts etc thereby accelerating your leanings and many more. And this is reflected in the library's growth rates too. Over the last few months, this framework has been growing at a rate of 200% and does not seem like stopping anytime soon.

This is why I was super excited to talk with Jerry and together we discussed how the project got started, the core concepts behind the framework, the mental models to help you wrap your head around LLMs and augment them with your own data, the teams vision for the project and how you can help out and join the movement!

jjmachanon I Launched my Product on Reddit to get 1,500 Sign-ups and 300k Page-views

how did you build you landing page. was it some online service or hand coded bootstrap theme

also thanks for the article, super helpful insights from your journey! All the best mate

jjmachanon Self-Driving Cars with Duckietown: Learning Autonomy on the Jetson Nano

man I'm really looking forward to attend this! love the community around this project and I'm actually hoping to create a similar one in my hometown too (Cochin)