If you are looking for ways to monitor LLMs which tie in with the rest of your observability stack, check out these docs [1] on monitoring LLMs with SigNoz. The tutorial with OpenLLMetry uses Otel format -
[1] https://signoz.io/docs/community/llm-monitoring/