Introducing Vector: Netflix's On-Host Performance Monitoring Tool
techblog.netflix.com
techblog.netflix.com
you should be able to install PCP from binary packages made available by the PCP development team on:
ftp.pcp.io
The directions _I_ saw for building from source looked pretty innocuous, but you might see a different set of directions if you're being MITMed. Observe an appropriate amount of caution.
The techblog isn't using SSL, and the git pull url for PCP is using the git protocol which is also unauthenticated, rather than the authenticated https transport (ssh is only an option when user accounts make sense).
Someone's at a conference and follows the link over public wifi. They get the same page but with "here's how to get PCP: ftp evil.io or git clone git://git.evil.io/pcp" Even if the webpage were ssl-enabled so that an attacker can't rewrite the pcp.io links, an attacker or evil network operator could MITM git.pcp.io or ftp.pcp.io. (FTP?!)
Being in Ubuntu's repo doesn't make it safe if Ubuntu's maintainers have no (semi-)trustworthy way of getting the code.
ftp://ftp.pcp.io/projects/pcp/download/MD5SUM
I've been researching for a rebuild of our ecommerce site with the idea of a modern microservice architecture. Team skillset and some other considerations dictate a .Net environment.
Netflix and other companies have created a really rich platform, between logging and monitoring technologies like this; message queueing; deployment; and so on. In the Windows environment there's precious little to compare to.
One bright spot is the news story also on HN today [1] about MS announcing Docker-related Hyper-V technology - but in the next version of Windows server.
It might be that to satisfy those .Net compatibility wishes, we just go to Mono and do everything else in Linux.
Then Vector should Just Work (tm) for Windows servers too.
Not to be a dick but... that doesn't sound like a Windows limitation.
In this one case the windows platform isn't going to hold you back too much. On top of that anything the .net stack on windows gives you access to WMI which can give you really simple really deep data from your application.
Don't buy into too much hype on what some of the big guys are doing on monitoring, most of their challenges seem to be around the scale they are monitoring at not with the depth of metrics.
One challenge is that there are some metrics that the kernel doesn't make available, so we need to either use tracers to fetch them, patch the kernel, or use custom kernel modules. In recent weeks, I've been using (or trying to use): ftrace, perf_events, SystemTap, eBPF, and LTTng. I've also used others, but not in the last few weeks. I'm really looking forward to getting these advanced tools into Vector.
Have you looked at any of the following for the various gripes you mentioned? I came from a primarily Linux background until recently, but I know .net and Windows have good tooling that extends beyond my modest list below. Looking to Microsoft to provide every option is probably not the best way to go about it, not sure if that was mostly what you were looking at so far.
https://www.rabbitmq.com/install-windows.html (company I work for uses it for messaging between their services)
http://nlog-project.org/ (we also use that as a base framework for our logging)
http://getakka.net/ (never used it on .net but can't do without it in the jvm)
https://github.com/dotnet/Orleans (sort of like Akka, but with additional features)
https://github.com/MSOpenTech/Redis (it's a bit behind the nix version, but still well maintained)
https://msdn.microsoft.com/en-us/data/gg577609.aspx (Netflix uses the Java port of that, fyi)
https://technet.microsoft.com/en-us/sysinternals/bb896653.as...
http://en.wikipedia.org/wiki/Resource_Monitor
https://www.jetbrains.com/dotmemory/
https://www.jetbrains.com/profiler/
http://www.appveyor.com/ (great for CI and easy to set up)
https://www.cygwin.com/ (when you miss a nix shell and tools)
http://en.wikipedia.org/wiki/Windows_PowerShell (for everything cygwin can't do easily)
I also don't think it's strictly true. For sure the underlying networking came from Unix. But as for the web itself, once we got to real dynamic content, it seems to me that Microsoft were the ones that got things moving.
While the Unix world was mired in the awful world of CGI, Microsoft gave us high-performance ISAPI, and then Cold Fusion (also on the Windows platform) and Microsoft's ASP made programming a little more sane. While the Unix world tried to deal with JSP (which IMHO wasn't a very good solution), the Microsoft platform seems to have been the innovators for several years, until Ruby on Rails and then node.js and stuff started coming out.
Today, the Apache server powers far more sites than any other, it's true. But IIS shares the 2nd-place spot [1]. When you say "There's a reason 80% of internet traffic does not use Windows", that's pretty much true for the servers, but far off the mark when counting clients. And the reason for that is that Microsoft's strength hasn't historically been radical advances, but in figuring out ways to take the bleeding edge tech that doesn't really work quite right yet, and packaging it into commodity software that may not be as sexy as envisioned by those with the original ideas, but actually useful to the average guy.
[1] http://www.zdnet.com/article/web-servers-microsoft-iis-and-n...
EDIT - missing word "world" in 3rd para
You claim IIS by using a ZDNet article from two years ago but the reality is IIS is number three behind Nginx and Apache. Still, being a distant #2 is nothing to brag about.
Now you're trying to claim clients are what powers the web but that's not the topic. What an amateur uses does not define what the professional uses. And to claim not wanting to be on the bleeding edge of things is no excuse for falling behind. Firefox and Chrome knocked IE off its perch years ago by being on the bleeding edge.
Stackdriver with pagerduty and I have 250,000 custom metrics being published and hundreds and hundreds of graphs on dozens of dashboards.
Although, I am looking at SignalFX to give even better version of this, but I manage nearly 1,000 machines with only a staff of four.
Until I noticed the big red flag on the front page "Stackdriver is now part of Google."
Surely Google is going to kill this product and use the knowledge to offer something similar for Google's cloud services.
I've spent more time trying to manipulate chart libraries into doing almost the same thing but just different enough to cause pain and suffering. Output your own path data and it's a million times easier.
For reference here's what I made: http://forrestthewoods.com/unbalanced-design-of-super-smash-...
Or if it's in-house, any info on whether they have, or might release it?
I see bootstrap-submenu.css mentioned, but not Bootstrap itself:
I've worked on production systems where everything else was failing (hardware, kernel, applications) but PCP kept chugging along, recording and telling the sad story to anyone that would listen.
Why not? Storage is cheap. Do you use something else to get historical visibility into metrics?
Vector is for per-instance custom drilldowns. I gave a talk last year where I showed how they both fit together: http://www.brendangregg.com/blog/2014-09-27/from-clouds-to-r...
I found it really easy to use.