Colony Graphs: Visualizing the Cloud (2014)

Seeing an entire subnet or fleet is fascinating. I hadn't seen the perf engineer side before, which is interesting. Process graphs are so fun!

We get involved in a lot of graph scenarios, and this one we frequently see in the security side, where folks are coming up with detections or investigating incidents. Imagine checking for shell calls with unexpected arguments, or a fleet not running as uniformly as expected. There was a post a few days ago about OSQuery -- combining these together opens a lot.

Edit: Meant to add -- we are launching louie.ai to 'talk' to your DB/SIEM/etc and get these kind of rich visual investigations, if of interest to anyone. Think notebooks, dashboards, orchestrations, API endpoints, etc . Again, we already expect this use case for security folks, but we are internally using louie.ai for our own general observability tasks, and I'd happy to onboard net eng etc folks wanting to do that too!

> My jaw dropped when I first saw this.

This is why cron tasks, always, should have some blocking (like flock) and/or timeout (like timeout from coreutils) and some kind of notification, better if it's something external to the server, like healthchecks.io

So cool!

Here's how to make them: https://www.brendangregg.com/ColonyGraphs/cloud.html#Impleme...

i've created graphs of arbitrary sizes and loaded the data into a graphing database for machines on the network.

by tracking network connections and configuration, you can build some really interesting graphs. It gets even more interesting if you overlay data, like roles or users.

With a graphing database it becomes easy to query and render N degree graphs of connected devices, pops, roles.

it'd definitely interesting approach to visualizing systems and networks and would recommend it to anyone

Reminds me of https://www.graphistry.com/

Related:

Visualizing All Processes in a Datacenter - https://news.ycombinator.com/item?id=15180048 - Sept 2017 (5 comments)

Why isn't there software that lets you visualize/zoom/interact with a live cloud status like this using D3? Or does Azure have something like this in its admin tools?