Shameless plug. We are building exactly this. Fully open source and open telemetry standard tracing with a visualization client that’s optimized for LLM application observability. Check it out here
- https://github.com/Scale3-Labs/langtrace - https://langtrace.ai/