It's not something that anyone with a choice in the matter should be using.
I guess this is something you don't notice as merely a "user", but IMHO it's horribly overengineered for what it does and I'm absolutely not a fan.
I also disliked the Go tooling for it, which is "badly written Java in Go syntax", or something along these lines.
This was 2 years ago. Maybe it's better now, but I doubt it.
In our case it was 100% a "tick off some boxes on their strategy roadmap documents" project too and we had much much better solutions.
OTel is one of those "yeah, it works ... I guess" but also "ewwww".
I'm not an expert in every language, but I am an expert in a few, and this just isn't something that you can assume. Languages like Go deliberately do not provide the sorts of features needed to support "automatic instrumentation" in this sense. You have to fold those concerns into the design of the program itself, via modules or packages which authors explicitly opt-in to at the source level.
I completely understand the enormous value of a single, cross-language, cross-backend set of abstractions and patterns for automatic instrumentation. But (IMO and IME) current technology makes that goal mutually exclusive with performance requirements at any non-trivial scale. You have to specialize -- by language, by access pattern (metrics, logs, etc.), by concrete system (backend), and so on -- to get any kind of reasonable user experience.
That is, until some open standard is defined by said Java astronauts.
I'm the founder of highlight.io. On the consumer side as a company, we've seen a lot of value of from OTEL; we've used it to build out language support for quite a few customers at this point, and the community is very receptive.
Here's an example of us putting up a change: https://github.com/open-telemetry/opentelemetry-js/pull/4049
Do you mind sharing why you think no-one should be using it? Some reasoning would be nice.
OTel lets the open source projects use an abstraction layer so that you can buy instead of self-host.
None of this has ever made me feel super great, but in the end I would probably consider OTel today for services that people other than my company operate. That way if some user wants to use Datadog, we're not in their way.
I used OTel in the very very early days and was rather disappointed; the Go APIs were extremely inefficient (a context.Context is needed to increment a counter? no IO in my request path please), and abstracted leakily (no way to set histogram buckets when exporting to Prometheus). I assume they probably fixed that stuff at some point, though.
More and more solutions are getting built in OTEL support, which means you can relatively seamlessly switch between backends without changing anything in your application code.
In the 2 jobs where I've set up the production environment, I just picked Prometheus/Jaeger/cloud provider log storage/Grafana on day 1 and have never been disappointed. You explode the helm chart into your cluster over the course of 30 minutes, and then move on to making something great (or spending a week debugging Webpack; can't help you with that one).
https://thenewstack.io/datadogs-65m-bill-and-why-developers-...
Their processors are quite capable and the entire receiver and exporter contrib collection is pretty good.
I'm not saying it's the best solution out there because that clearly depends on each use case but I don't think such harsh criticism makes sense.
Disclaimer: I'm part of the fluent-bit maintainer team.
From an ops point-of-view devs can add whatever observability to their code and I can enforce certain filtering in a central place as well as only needing one central ingress path that applications talk to.
Also because everything emits OTLP if we ever want to move to new backends it's just a matter of changing a yaml file and not rewriting applications to support a new logging backend.
Given the choice of going back to the old way of using vendor-specific logging libraries, I will continue using OTEL 10/10 times because even given its warts, it's still a lot nicer than the alternatives.
You can't build a sound product if that's one of the design requirements.
Switching to a new backend is as simple as deploying the new backend, changing 1 line in the OTEL Collector yaml, then having your front-end pull from the new backend. 0 changes to application code necessary.
These domain concepts are descriptive, not proscriptive. They don't, and can't possibly, have specific wire-level definitions. So another way to phrase my point might be to say that OTel is asserting definitions which doesn't actually exist.
Telemetry necessarily requires specialization between producer (application/service) and consumer (observability backend) in order to deliver acceptable performance. It's core to the program as it is written: more like error handling than e.g. containerization.
With basic parameters in place so it doesn’t eat your billing, it’s been working great with me for years. Initially with New Relic, then Datadog, now a setup with OpenTelemetry is good enough.
You should "use" metrics, logs, and traces thru dependencies that are specific to your organization. The interface between your business logic and operational telemetry should be abstract, essentially the same as a database or a remote HTTP endpoint or etc. The concrete system(s) collecting and serving telemetry data are the responsibility of your dev/ops or whatever team.
Main point: instrumentation is part of the development process, not something that's automatic or that can be bolted-on.
There is no one true interface! The interface is a function of the sub-class of telemetry data it serves, the specific properties of the service(s) it supports, the teams it's used by, the organization that maintains it, etc. etc.
OTel tries to assert a general-purpose interface. But this is exactly the issue with the project. That interface doesn't exist.
I recall Apache Skywalking being pretty good, especially for smaller/medium scale projects: https://skywalking.apache.org/
The architecture is simple, the performance is adequate, it doesn't make you spend days configuring it and it even supports various different data stores: https://skywalking.apache.org/docs/main/v9.5.0/en/setup/back...
The problems with it are that it isn't super popular (although has agents for most popular stacks), the docs could be slightly better and I recall them also working on a new UI so there is a little bit of churn: https://skywalking.apache.org/downloads/
Still better versus some of the other options when you need something that just works instead of spending a lot of time configuring something (even when that something might be superior in regards to the features): https://github.com/apache/skywalking/blob/master/docker/dock...
Sentry comes to mind (OpenTelemetry also isn't simpler due to how much it tries to do, given all the separate parts), compare its complexity to Skywalking: https://github.com/getsentry/self-hosted/blob/master/docker-...
I wish there was more self-hosted software like that out there, enough to address certain concerns in a simple way on day 1 and leave branching out to more complex options like OpenTelemetry once you have a separate team for that and the cash is rolling in.
Internally, OTEL has to keep track of every combination of labels it's seen since process start, which can easily come to dominate the processing time in an existing project. It's another in a long line of tools that dovetail with my overall software development philosophy which is that you can make pretty much any process work for 18 months before the wheels fall off.
By the time you notice OpenTelemetry is a problem, you've got 18 months of work to start trying to roll back.
I'm sure there are valid qualms with OTEL in general, but this ain't one of them. Any and all metrics telemetry systems can fall into the same design constraint you pointed out.
Push implementations do not have this problem at the client end.
Well, every unique combination of labels represents a discrete time series of telemetry data, and the total set of all time series in your entire organization always has to be of finite and reasonable cardinality. This means that label values always have to be finite e.g. enumerations, and never e.g. arbitrary values from user input.
> my overall software development philosophy which is that you can make pretty much any process work for 18 months before the wheels fall off.
The size of the set of labels in your process after (say) 1d of regular traffic should be basically the same size as after (say) 18m of regular traffic. If this isn't the case, it usually signals that you're stuffing invalid data into label values.
> OpenTelemetry is a collection of APIs, SDKs, and tools. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior.
You can absolutely categorize telemetry into these high-level pillars, true. But the specifics on how that data is captured, exported, collected, queried, etc. is necessarily unique to each pillar, programming language, backend system, organization, etc.
That's because telemetry data is always larger than the original data it represents: a production request will be of some well-defined size, but the metadata about that request is potentially infinite. Consequently, the main design constraint for telemetry systems is always efficiency.
Efficiency requires specialization, which is in direct tension with features that generalize over backends and tools, e.g.
> Traces, Metrics, Logs -- Create and collect telemetry data from your services and software, then forward them to a variety of analysis tools.
and features that generalize over languages, e.g.
> Drop-In Instrumentation -- OpenTelemetry integrates with popular libraries and frameworks such as Spring, ASP.NET Core, Express, Quarkus, and more! Installation and integration can be as simple as a few lines of code.
I think OTel treats these goals -- which are very valuable to end users!! -- as inviolable core requirements, and then does whatever is necessary to implement them. But these goals are not actually valid, and so the resulting code is often inefficient, incoherent, or even unsound.