mmclean

Likely Prometheus - Jaeger for tracing, Elastic for logs, Prometheus for metrics is a pretty common and effective OSS observability stack

Some of their engineers have made PRs to the OpenTelemetry JS repository, but it wouldn't have been mature enough (or even existed) when much of the work described in the blog was underway.

Regarding Google Cloud Profiler (I'm the PM), this is for a few reasons:

- We have good support for Go, Java, JS, and Python, but are still adding a few features for these languages (MUSL support for Alpine just shipped, still need to add heap profiling for Python)

- C++ isn't as heavily used by our ops tools customers as Go, Java, JS, and Python

- We have several new analysis features in the pipe, and the cost supporting an additional language would slow down the delivery of these

There's no lack of desire to add new languages, but we chose to prioritize completing existing language support and new analytics functionality this year. I'm guessing that the other teams making products in this space face similar constraints and made similar tradeoffs.

Existing tracing systems (Dapper, Zipkin, Dynatrace, Stackdriver, etc.) already randomize with each hop, and there was a desire to be consistent with the models that they already used. It's also more straightforward to implement.

There's a discussion about "correlation context" inside of this W3C group called , which maps to what you're describing. It'd be worth reaching out to Sergey (one of the other co-chairs) if you want to find out more.

FYI for all, the W3C TraceContext specification will become a Proposed Recommendation later this week. I'm one of the co-chairs of the group and am happy to answer questions about our W3C work or OpenTelemetry.

W3C Trace Context defines an HTTP header format for traced requests, and OpenTelemetry implements this format by default. While this project is technically distinct from OpenTelemetry, it's effectively composed of the same people (including me).

Right now we're focused on the first release of OpenTelemetry, which will include distributed traces and metrics. Many users and contributors have asked for logging support, and this was already a big discussion topic in the pre-merger OpenCensus and OpenTracing communities - I'd expect us to start focusing on this after the release later this year. There have also been some early conversations around support for an error first-class signal type (there was a recent GitHub issue, though now I can't find it), however to my knowledge we haven't yet started any error-related specs or design discussions.

You can certainly add error-related annotations to traces, however these will typically be sampled.

Ah, got it. We've discussed exactly this (within Google) with some of the Kubernetes maintainers who have similar wishes to yourself. The most immediate z-pages related work is focused on making them better in the more general case, as they're extremely barebones at the moment and are specific to each language's library. After that we'll explore the native Kubernetes integrations that you're asking for.

It'll come, but it's going to take some time.

Yep! The Java library has some basic z-page support, but adding high quality z-pages will be a big part of the OpenCensus agent and OpenCensus service that we're developing now. These can be used within Kubernetes environments or on plain VMs.

You can send traces to Jaeger, Zipkin, Stackdriver, Azure App Insights, X-Ray, Honeycomb, and a few others. Supported metrics backends include Prometheus, Datadog, SignalFx, Stackdriver, Azure, etc.

Another OpenCensus maintainer here, happy to answer questions along with Nik

[SD APM PM here] Yep, we support Java / JVM, Node.js, and Go; Python is coming soon

FYI, OpenCensus doesn't lock you in to a specific provider, rather it includes its own tracing implementation and exports traces back to your provider of choice

OpenCensus [1] provides an implementation for tracing and app-level metrics, a context wire protocol, and exporters for various backends. While it provides similar tracing APIs to OpenTracing, they're not identical, though there have been some early conversations about resolving this.

I'm the PM for Census at Google, though other vendors are involved as well. Basic implementations for Java and Go are already functional, with much more to come. We gave a small talk at Kubecon last month, but will start promoting the project once it's slightly more mature.

[1] http://opencensus.io/

They're different in scope and goals. Census is composed of a context propagation format (which the blog post mentioned), a single distribution of language-specific libraries that include instrumentation hooks for popular web / RPC frameworks and metrics + trace exporters for various backends, and an optional agent that you can run locally to view metrics and RPC stats.

PM for tracing at Google here.

As the blog post mentioned, we're collaborating with partners on a common RPC context proposal for tracing systems [1]. Some of the major contributors are currently on vacation, but there'll be be another round of comments and updates once they get back.

We're also working with the same partners on a project called Census - a set of tracing instrumentation libraries that all vendors can use and contribute to, though we're still in the very early days on this effort. We'll have more to announce later in the year (still need to publish additional libraries, set up a website, etc.), but you can follow the progress on GitHub here [2].

[1] https://github.com/TraceContext/tracecontext-spec/pull/1/fil... [2] https://github.com/census-instrumentation

Yep, an issue with the anchor links that'll be updated soon. It's supposed to just link to a lower part of the page.

If you scroll down you'll see a top-level header also titled "Configure Zipkin tracers" that describes how to configure the Brave tracer. You don't really need to do anything special here - just point your Zipkin tracers at the Stackdriver Zipkin Collector rather than your existing one.

Nevermind, I think we found it. Fixing now.

Uh oh - are you referring to the docs on GitHub or the actual docs site (https://cloud.google.com/trace/docs/zipkin)?

Stackdriver Trace [1] is Google's external implementation of Dapper with additional analysis features. It's available for free, even for workloads not running on Google Cloud Platform.

(Disclosure: I work on it)

[1]: https://cloud.google.com/trace/