Hacker News Reader

Comment by streblo | Hacker News Reader

I'm hugely disappointed with OpenTelemetry. In my experience, its an over-engineered mess and the out-of-the-box experience is super user hostile. What it purports to be is so far away from what it actually is. Otel markets itself as a universal tracing/metrics/logs format and set of plug and play libraries that has adapters for everything you need. It's actually a bunch of half/poorly implemented libraries with a ton of leaky internals, bad adapters, and actually not a lot of functionality.

View on HN

Agreed, I find myself having to think orthogonally to common sense whenever I try to use one of its SDKs. Nothing works the way you expect it to, everything has 3 layers of unnecessary abstraction and needs to be approached via the back door. Many features have caveats about when it works, where it works, how much it works, during what phase of the moon it works and how long your strings can be when Jupiter is visible in the sky.

That said, if we disregard the leaky SDK APIs and half-implemented everything, it does somewhat deliver on the pluggability promise. Before OTel, you had bespoke stacks for everything. Now there is some commonality - you can plug in different logging backends to one standard SDK and expect it to more or less work. Yes, it works less well than a vertically integrated stack but this is still something. It enables competition and evolution piece by piece, without having to replace an observability stack outright (never going to be a convincing proposition).

So while the developer experience is pretty unpleasant and I am also disappointed with the actual daily usage, from an architectural perspective it opens up new opportunities that did not exist before. It is at least a partial win.

Yes, I really agree, and I've gone through the same pain, but try using the alternatives that claim to be better because they have OpenAPI specifications [1]

The example shows you how to use the swagger tool, parse the OpenAPI spec [2], auto-generate GoLang glue code, call __one__ of those auto-generated functions and log a trace.

However, there is zero documentation, zero other examples, and I'm left scratching my head whether there's even one person in the world using this approach. I eventually ended up just directly using the service APIs [3] via REST calls.

OTEL is painful, but the alternatives are no better :( I really wish there's some interest in this space, since SLO's and SLI measurements are becoming increasingly important.

[1] https://github.com/openzipkin/zipkin-api-example

[2] https://github.com/openzipkin/zipkin-api/blob/master/zipkin2...

[3] https://zipkin.io/zipkin-api/#/

Prometheus text exposition format is de-facto standard used in monitoring. It would be great building an official observability standard on top of it. This format is much easier to debug and understand than OpenTelemetry for metrics. It is also more efficient, e.g. it requires less network bandwidth and less CPU for transfer than Otel for metrics.

[1] https://github.com/prometheus/docs/blob/main/content/docs/in...

Ok, then do you have a suggestion for an alternative, or do you just put up with OT?

I agree with OP, but I put up with Otel

Okay but can you point to some specific experience and suggest improvements?

Try implementing an OTEL Tracer. This interface is insane, and should really just be a struct.

https://github.com/open-telemetry/opentelemetry-go/blob/trac...

Why were you implementing your own tracer? Don't they publish an implementation?

Yes, all SDKs have a tracer you can just use. While you can technically create your own tracer, you're officially in Hard Mode territory - there's no highly extensible system I'm aware of that makes swapping core concepts (that already have a default) easy.

how is this interface insane?

It's a list of the behaviors you need to implement if you're rolling your own OTEL Tracer Span implementation, and not using one of the multiple available.

In contrast, OpenTracing's interfaces had hardly any required methods, so you had to do a runtime type-cast to the whichever implementation you were using in order to access anything useful on the Span like the OperationName.

Reply on news.ycombinator.com