AWS X-Ray – Distributed Tracing System
aws.amazon.com
aws.amazon.com
As a starting point, I would recommend reading Google's paper on their project "Dapper." [1] It's essentially the core of most distributed tracing systems. At least those I've encountered.
There's a lot of tooling out there that take their cues from Dapper. I've recently been looking into integrating OpenZipkin[2] with our systems. I see at as a more viable alternative (no tie in!) to Yet Another Propriety Thing in AWS (YAPTA). There's other as well, like AppDash.
Recently there has been a push towards an open-standard for the collection side called OpenTracing[3]. I came across it when investigating LightStep[4]. Ideally that means no vendor lock-in, which of course has lots of knock-on effects.
If you know of any, I'd love to be pointed in the direction of _different_ and not just divergent techniques.
[1]: http://static.googleusercontent.com/media/research.google.co...
[2]: http://zipkin.io
[4]: http://lightstep.com - Impressive team behind this.
and their follow on work, Pivot Tracing: http://pivottracing.io/
(which got a best paper award at SOSP'15.)
In practice, the analysis a DAG allows you to perform are way more powerful. So if you're in the process of setting that request ID stuff up, consider doing the tracing way.
(Disclosure: I work on it)
It doesn't show zipkin configuration :)
If you scroll down you'll see a top-level header also titled "Configure Zipkin tracers" that describes how to configure the Brave tracer. You don't really need to do anything special here - just point your Zipkin tracers at the Stackdriver Zipkin Collector rather than your existing one.
(I work on it)
I'll be talking about this on the https://twitch.tv/aws stream at 12:30 pacific if you guys want to ask questions / learn more.
(I WORK AT AWS)
There are still a lot of talented and passionate people at Amazon who want to execute on other products. The S3 team is still growing 10 years later and they're still doing (IMO) cool stuff. I'm a devangelist though so I don't always get to see the inside of every team (not enough time). I do know that we all drink the customer focused kool-aide and we genuinely believe in it. Sorry if this all sounds like marketing.
AWS's crustiness is exactly what I would expect from an agglomeration of adversarial teams each looking to minimize their own liability at the expense of everyone else. Given that they're still organized that way, the surprise would be if they turned the trend around, not if it continued.
As to some concerns about diluted focus, service teams operate with great deal of autonomy. From what I have seen, features are driven by teams, and often as a result of conversations with customers.
I'm probably over thinking it but big bang announcements of multiple products does make me wonder.
That might or might not be a downside to you.
I don't want to appear a kook, so I won't relitigate what I said in another reply. But essentially we do want to give back. We just aren't always able to give back everything (at least not right now).
AGPL, like GPL, is all or nothing in that way. I don't get the option of giving back something, then giving back more later (OSS is threatening to some people - I have frequent discussion at work re why we give anything at all back).
Your software, your rules. You have every right to take this all or nothing position! More power to you.
But assuming that anyone that doesn't like AGPL does so because they "want to use someone else's work while hoarding [their] own" is, respectfully, a simplistic and stereotyped point of view.
They worry that if they use an AGPL javascript library for the video player on their front page, they'll have to opensource the whole web application (I've even sat in on debates over weather using GPL3 programs means you have to opensource any source data you use them to process and publish the result).
A lot of this is silly, but a lot of it isn't. A large company I worked for stopped using Linux after they were forced by a court to opensource a custom video processing microchip because they ran Linux on the box and drove the chip through a kernel driver.
It sounds like a very obvious choice when you phrase it as "we wouldn't contribute patches to you", but that's just a way of holding out a carrot and hoping for people to bite, phrasing it as a "your loss" scenario. At least it is, to me. What am I losing, exactly? My code is right there. I haven't lost anything at all, except maybe some vague, hypothetical possibility of a little extra help -- which, in general, seems to rarely materialize in the way people imagine anyway, for a host of reasons (companies are slow, they're lazy, their hacks are incomplete and unsuitable for an upstream, they're anti-OSS contribution in general and only disallow using the GPL, but disallow contributing to anything, etc).
There is plenty of software that, very arguably IMO, should be available under non-GPL terms for a host of reasons, it's true (even the FSF agrees with this, to help support and promote e.g. interoperable, royalty-free codecs, algorithms, which benefit from public domain availability). And licensing is also a complicated legal, political and, at some level, personal topic as we pick licenses that we believe represent our goals and values. We should treat it that way.
But phrasing the scenario in such a manner, where there's a narrative that "The only thing the AGPL will do is ensure I write 0 patches (even though I would have likely written 0 patches in any case), so uhhh, think about that", feels like an attempt to poison the well, even if you didn't intend it to be. We could just as easily turn it around: if you literally aren't going to contribute patches, to return some of the benefit bequeathed to you by the virtue of using my code -- why should I even care a tiny bit about you or your company, or whether you "need" to use my software to achieve your own goals? Clearly, you don't need it so badly that you're willing to agree to the rules. Why bother? So you can just freeload on the people who do agree to the rules?
If this isn't your intent or I'm being overly presumptuous, I apologize. But even then, I still dislike the narrative that pops up around these arguments, even if I may be reading too much into your particular comment.
In a typical scenario for us, we use some open source software for our SaaS platform. We need to extend it in ways that weren't envisaged, so we (a) add some hooks to allow extensions and (b) add the extensions themselves.
Our preference is to contribute both (a) and usually (b) back to the project. But sometimes for business reasons we can't contribute (b), or at least not until more people internally have their heads around it.
AGPL stops us from contributing (a) and not (b). So we just don't use AGPL software.
I wasn't trying to poison the well, I was just indicating what our decision making process is. Some people, like us, have valid business reasons for not using AGPL software, even though we love OSS.
I find it weird that this narrative thing pops up here. There's no narrative. If I was the owner of said AGPL software, I probably wouldn't give a rat's arse if anyone said they weren't going to use my software.
But my original post was a specific response to someone who said "Not sure what the downsides are." [of using AGPL].
So let me try again. Here's one possible downside - *you may miss out on some patches that you would have otherwise got, since your license is incompatible with some people's business needs".
Almost from 2 years ago:
https://news.ycombinator.com/item?id=9358843
But gist is, Amazon employees are generally discouraged from contributing to open source at all.
It makes business sense even if it's a shitty thing to do. The stronger an open source project is that provides the core functionality that your managed service provides the easier it is for a competitor to build the same thing. Amazon isn't as cutthroat as Oracle but they're certainly much less OSS friendly than many of their competitors.
To provide a performant and cost-effective experience, X-Ray does not collect data for every request that is sent to an application. Instead, it collects data for a statistically significant number of requests. X-Ray should not be used as an audit or compliance tool because it does not guarantee data completeness.
I imagine this being used for episodic debug cases, where one could turn up the sample rate and pay only for the traces captured/queried during an incident. But you would need to do your monitoring and trending separately.
I wouldn't be surprised if they eventually start integrating this better with CloudWatch for that reason, though it doesn't seem to be doing any of that today.
Disclosure: I work on TraceView (traceview.solarwinds.com) which is distributed tracing based APM product. We're inspired by Google Dapper and x-trace, both mentioned elsewhere in this thread.
I don't see the distinction you're making, as neither seems to record all traces for accurate audit. Am I missing it?
If you're looking specifically for the 100% audit trail case, I'd look at DynaTrace or potentially Instana.
The service works via the SDK's in various languages, that report tracing information to a local daemon that runs on the host, over UDP.
The daemon then batches the data, applies sampling (Which is configurable, all the way to 100% - report everything), and sends it en-masse to AWS.
Edit: See Sampling Rules section here - http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-nod...
I concur, it was specifically stated that 100% sampling is an option (and is by default?)
> You can use X-Ray with applications written in Java, Node.js, and .NET that are deployed on these services. Support for AWS Lambda is coming soon.
> AWS X-Ray supports tracing for applications that are written in Node.js, Java, and .NET.
Edit: The blog post mentions they will be adding this to the other SDK's.
https://reinvent.awsevents.com/
Google, Apple, Microsoft, and Amazon each dominate the frontpage for several days when their respective developer conferences are ongoing or just completed.
Those products are primarily based on code profiling. For example, at Stackify we automatically profile key methods for dozens of common dependencies and frameworks to understand their usage and performance. Every SQL, NoSQL, caching, queuing providers and many other things. Plus app errors, logs, etc.
So the best I can tell from the AWS blog and docs is the answer is no its not a full APM. It appears to track how long a web request takes and any usage of the AWS services via their SDK. More of a lightweight service mapping of AWS services. So SQL database or HTTP calls probably aren't tracked.
In the future could they expand it? Sure. But for now it seems limited compared to a full blown APM product. Although this could be help for identifying performance problems with AWS services.
Matt - Founder of Stackify
X-Ray seems like a pretty basic service at this point but in my opinion the writing is on the wall for other APM providers. Amazon chose this announcement for a major slot so I expect they'll be investing in the service. As Bezos is often quoted as saying "your margin is my opportunity."
https://azure.microsoft.com/pl-pl/services/application-insig...
If there's no Python support - that shocks me.
The similarity to Dynatrace is that - like the PurePath - X-Ray enables distributed tracing. But the approach is different: whereas Dynatrace instruments your application using an agent^, X-Ray needs you to instrument an application using the provided SDK. Moreover, X-Ray seems to be limited to applications communicating via HTTP(S).
^ in the most cases