Hacker News Reader

Congrats on the launch! Curious to learn what specific use case you have seen around observability of LLM apps which are not covered by standard observability tools like DataDog, SigNoz, etc

Also, how do you compare in terms of features with DataDog's LLM monitoring product which was launched recently?

Disclaimer : I am a maintainer at SigNoz

View on HN

This is Max, one of the co-founders. We appreciate existing observability tools as they have saved us so much time in the past already. Excited to get your view on this! We've found many observability demands to be quite different when working on LLM applications. Mainly: Unpredictable input (users input free-form text that cannot be fully tested for), control flow highly dynamic when running on the textual output of a previous step and quality of output is not known at runtime (for the application it is just text). Many teams read manually through the LLM inputs and outputs to get a feeling for correctness or ask for user feedback. In addition, currently working on abstraction for model-based evals to make it simple to try which one works best for a use case and automatically run it on all production prompts/completions. One user described the difference to be that they use observability usually to know that nothing is going wrong whereas they use Langfuse many hours per day to understand how to best improve the application and navigate cost/latency/quality trade offs.

Reply on news.ycombinator.com