Hey! On the metrics front, we do basic metrics like requests, tokens and cost for now as all of the usefull information about llm is included directly in the Span Attributes of the Trace.

So what a lot of users have told me is the want to track their usage(cost/tokens) and want to keep an eye on the user interactions like prompts and responses, We are looking to add more accuracy based metrics too which is very highly asked to us.

Also re on UI: UI is optional, You can use the sdk to directly send otel traces and meyrics to your preferred destination. https://docs.openlit.io/latest/connections/intro