Hacker News Reader

Comment by cedws | Hacker News Reader

I still don’t understand what OTEL is. What problem is it solving? If it’s a standard what is the change for the end user? Is it not just a matter of continuing to use whatever (Prometheus, Grafana, etc) with the option to swap components out?

View on HN

For the tracing part of Otel, neither Prometheus nor Grafana are capable of doing that. Tracing is the most mature part of Otel and the most compelling use case for it. For metrics, we've stayed with Prometheus and AWS Cloudwatch Metrics. The metrics part feels very under developed at the moment.

When I last looked 9 months ago, there were libraries of the metrics side of the tree still marked as experimental, that you couldn’t successfully send metrics without using. And a huge memory leak in the JS implementation that was only fixed 15 months ago: https://github.com/open-telemetry/opentelemetry-js/issues/41...

Things, especially crosscutting concerns, you want to use in production should have stopped experiencing basic growing pains like this long before you touch them. It’s not baked yet. Come back in a year. Or two.

Everything is either in development or stable. There aren't statuses like alpha, beta, release candidate, etc. except for individual library releases. Metric clients will be marked as "development" until it goes "stable" [0]. Consequently it can be hard to determine the actual maturity level of any given implementation.

Tracing is very mature, with metric and logging implementations stable for a number of popular languages [1].

the "experimental" status was renamed "development"

[0] https://opentelemetry.io/docs/specs/otel/versioning-and-stab...

[1] https://opentelemetry.io/docs/languages/#status-and-releases

> the "experimental" status was renamed "development

That doesn’t really change things now does it. It’s still a bunch of people sitting around saying “MMMM” loudly while eating half-raw cookies.

The point of OTel is interoperability.

For example the author of the software instruments it with OTel -- either language interface or wire protocol -- and the operator of the software uses the backend of choice.

Otherwise, you have a combinatorial matrix of supported options.

(Naturally, this problem is moot if the author and operator are the same.)

Interoperability with what?

Where are the three existing, successful solutions it is trying to abstract over?

It doesn’t know what it is because it’s violating the Rule of Three.

Interoperability with the other things your Otel Vendor is selling you. No two implementations are even remotely compatible, but they can all mostly scrape data from your Prometheus endpoints, so it's easy to migrate from useful software to their walled garden.

> to their walled garden

Am I detecting sarcasm or did I just bring my own?

I don't think there's any sarcasm there; Perhaps a wistful hope that behind one of those walls somebody's actually got a garden instead of just a seedbed of false promises.

Yarp.

Application Insights, Data Dog, New Relic, etc…

APM products in general.

How to send Prometheus data to New Relic: https://docs.newrelic.com/docs/infrastructure/prometheus-int...

How to send StatsD data to Datadog: https://docs.datadoghq.com/developers/dogstatsd/?tab=hostage...

Places like datadog and posthog are selling you their ability to ingest your existing data. I call bullshit. It’s a problem looking for a solution. It’s an excuse for engineers to build moats around a moderately difficult problem by making it inscrutable.

It simplifies the issue for applications and libraries that can support Otel and quickly integrate with whatever is being used to collect the data.

The whole idea of OTel is not to be a moat.

It's the exact opposite.

I can export traces (or metrics or logs) to whatever backend I want, and change easily.

You’re talking about a data moat, I’m talking about intellectual moats and intentional complexity, as opposed to accidental or essential complexity.

You're ignoring the oldest and most mature part of OTEL which is traces.

If you look up how to send traces to any popular vendor the options are either a) use our proprietary for at and proprietary agents and SDKs, or b) use otel

Iirc metrics in OTEL are very similar to Prometheus. Haven't looked at logging but realistically logging becomes an afterthougt with a good tracing setup.

interoperability between vendors, so your business isn't stuck with a vendor who can raise prices because their SDKs are deeply embedded in your codebase, so open source libraries / products have a common point to hook into without needing to integrate with each vendor.

> Interoperability with what?

For the backend?

Datadog, New Relic, Grafana, Sentry, Azure Monitor, Splunk, Dynatrace, Honeycomb

Aside from what others have mentioned (being interop on the send and receive side of things) I find a great deal of value in the interop schema, so you don't have to re-learn what every joker wants to name their special unicorn flavor of kubernetes pod name or container id or http response code when exploring metrics or making dashboards

https://opentelemetry.io/docs/specs/semconv/general/attribut...

https://opentelemetry.io/docs/specs/semconv/hardware/common/

https://opentelemetry.io/docs/specs/semconv/system/container...

https://opentelemetry.io/docs/specs/semconv/system/k8s-metri...

https://opentelemetry.io/docs/specs/semconv/http/http-metric...

https://opentelemetry.io/docs/specs/semconv/cloud-providers/...

This is worthy of a bookmark. (If HN supported bookmarks.)

https://news.ycombinator.com/item?id=42658095#:~:text=favori... and then https://news.ycombinator.com/favorites?id=PeterCorless&comme...

Thanks! TIL.

You can favorite comments. (And posts.)

i can report the same traces to jager if i want open source or i switch out the provider and it can go to aws x-ray (paid). without any code or config changes. pretty useful. yes, a tad clumsy to set up the first time.

Reply on news.ycombinator.com