Instrumentation of a given system is by definition always "larger" than the nominal traffic which flows through that system. That is, given some request R, the possible observable metadata of that request R is essentially infinite. So when you're designing an observability system, the whole ball game is about capturing and constraining the cardinality of that metadata. Observability as a field is _entirely_ about reducing the volume of telemetry data such that it can be usefully captured, aggregated, and queried by operators. That's why we have metrics, traces, and logs as distinct things. They're all optimizations of the same underlying information, each optimized for different consumer use cases.

Concretely: if user U1 does an HTTP GET to /spotify/track/123, that's perhaps 10KB of production traffic, but easily 10MB of telemetry traffic. In effect that telemetry can be modeled as a huge key/val map of metadata, but you can't do it that way and remain efficient, you have to optimize for observability use cases. You have to increment a cardinality-bound set of metric counters for the request outcomes, and maybe emit some best-effort trace data for the request ID, and etc. etc. — _as separate things_!

The engineering costs dominate the design. But OpenTelemetry says this isn't the case. OpenTelemetry says that whatever requirements are on FX00 company CTO feature checklists are valid a priori, and commits to doing whatever is necessary to satisfy them. That's because OpenTelemetry is evaluated not on any technical merits, but on the adoption rate of the CNCF stack among those FX00 companies.

OpenTelemetry is explicitly and exclusively a thing meant to tick off an "observability" checklist item on the checklist of a FX00 CTO's due diligence form. That's it. That's the only goal. Nobody with a choice should be using it. Read the source code, it's abysmal.

Alternatives? Write code that leverages each pillar of observability directly. There's no short-cut. That's the whole point.