Hacker News Reader

Comment by verdverm | Hacker News Reader

This seems limited compared to the observability frameworks that integrate with the popular LLM frameworks like LangChain and LlamaIndex. When you use those, you want automatic traces for the nested calls hidden down the function call chain.

This really seems more like a Flask API tracing example with as LLM call as an endpoint. Dare I call it a shallow developer marketing piece?

re: LLM observability more generally, they mostly seem to be building their own full-stack suite, whereas I'd like them to ship to my existing Grafana LGTM stack. I'm keen to check out https://docs.openlit.io/latest/introduction and migrate away from LangFuse, but there is the feedback on model response -> training dataset that is really nice

View on HN

If you are looking for ways to monitor LLMs which tie in with the rest of your observability stack, check out these docs [1] on monitoring LLMs with SigNoz. The tutorial with OpenLLMetry uses Otel format - [1] https://signoz.io/docs/community/llm-monitoring/

I'm using Garfana LGTM, not SigNoz.

HN generally appreciates if you account for this and only promote your project when it's relevant to the discussion.

If you are looking for automatic instrumentation that generates open telemetry standard traces - check out Langtrace, https://github.com/Scale3-Labs/langtrace. Got support for all popular LLMs, frameworks and vectorDB with high cardinality.

This still has the same issue that the docs say I need 2 more databases, not making use of my existing grafana LGTM stack

https://github.com/Scale3-Labs/langtrace?tab=readme-ov-file#...

Another blocker for me is that it appears to need an API key for trying it out locally or self-hosting

---

I'm comparing your project to OpenLit on this front

- https://docs.openlit.io/latest/installation#kubernetes

- https://docs.openlit.io/latest/connections/grafanacloud

1. You do not need 2 databases if you are only looking to export the traces to your grafana stack. If you are using prometheus as the TSDB, you can directly send the traces there without needing any of the databases to be setup locally.

2. Also, you do not need the API key. You only need to install the python or typescript sdk and use the custom exporter option - https://github.com/Scale3-Labs/langtrace-python-sdk?tab=read... . If you already have a OTEL exporter running, you don't need even that. Just initialize the sdk after installing and it will do its thing.

I am one of the core maintainers of the project. Do let me know if you have any questions.

Reply on news.ycombinator.com