Hacker News Reader

Comment by swyx | Hacker News Reader

congrats on launch!

the thing about OTel is that it is by nature vendor agnostic. so if i use OpenLLMetry, i should be able to pipe my otel traces to whatever existing o11y tool I use right? what is the benefit of a dedicated monitoring platform?

(not cynical, just inviting you to explain more)

View on HN

Great question and I see you already got a similar answer but I'll add some of my thoughts on this. We are actively promoting OpenLLMetry as a vendor agnostic way of observing LLMs (see some examples [1], [2]). We believe that people may start with whatever vendor they work with today and may gradually shift or use something like Traceloop because of specific features we have - for example the ability to take the raw data that we output with OpenLLMetry and add another layer of "smart metrics" (like qa relevancy, faithfulness, etc.) that we calculate on our backend / pipelines; or better tooling around observability of LLM calls, agents, etc.

[1] https://docs.newrelic.com/docs/opentelemetry/get-started/tra...

[2] https://docs.dynatrace.com/docs/observe-and-explore/dynatrac...

Not OP here (but building in the same space). The reason you instrument LLM data is usually to improve quality/speed of your applications. The tools to extract the insights to enable that, and the integration with your LLM experimentation workflow is the differentiator between a general observability solution and LLM specific one.

oh cool. do you also consume OTel? or something else?

Right now we have our own instrumentation but we're working towards Otel compatibility.

Reply on news.ycombinator.com