Internet-monitoring – A Docker stack that monitors your home network
github.com
github.com
I saw a comment below where some was rolling their eyes that you "complicated" stuff with Prometheus, Grafana, and Docker and how you could just use Bash scripts and crons. As I just upgraded my codebase from this more bare metal approach to this "more complex setup" I'd like to mention: there's no way you could do time series statistical analysis easily with "just a cron job and a bash script". Prometheus and Grafana are for more than just buzz words. Prometheus offers an advanced time series database which allows you to, at minimum, do more robust analysis using data techniques like Histograms. As for Grafana, it makes exploring data dead easy.
Providing users with a Docker Compose setup is also something I did with my tool and the benefits are huge. It lets me distribute a setup which relies on multiple moving parts working smoothly together. Sure I could write a whole wiki on how you should setup Prometheus Grafana and my tool, or I could distribute the setup with a configuration as code tool. Ensuring that even if someone doesn't want to use Docker Compose they can at least read my configuration as code and see exactly what I did to setup my tool.
Looking for alternatives, I found InfluxDB, which was just one apt-get install away.
It comes with a web interface to create dashboards, supports push based metrics, and overall I'm loving it.
The web interface doesn't seem as powerful as grafana, but it covers most basic needs. And I think you can use it with grafana too.
You can indeed; I use InfluxDB and Grafana to provide some stats on my home server. Pretty easy to set up, and Grafana just uses the Flux query language to pull the data from Influx.
If you are on a ASN that they don't have many existing probes for, they'll send you one for free.
Thanks for sharing that link for Ripe though, super cool project.
E.g: i struggled figuring out where to put a non-standard probe
Failure modes were also quite weird... adding a slave to the mix broke my mind a little. Was never sure it worked but then didn't need it so retired it before i figured it out
Changing what you think is something trivial (e.g: frequency of data collection) invalidates your previously collected data files.
Other than that - fairly solid, did what i needed, runs without a fuss.
I used a docker container, and it was a breeze! `sudo docker run -it -p 8000:80 -e WIPE=y -e TARGET="ISP;NextHop;$IP" -d dperson/smokeping`
This project is quite good for a smokeping-a-like for prometheus.
Caveat: since you're probably running OpenWrt on a consumer SoHo router, I recommend sending this logging data to a cheap USB flash drive, not wearing out the router's onboard flash.
An even simpler alternative would be to keep a browser window open with the OpenWrt admin UI's "Status -> Realtime Graphs" displaying.
(If you're new to OpenWrt, a fairly recent router hardware model that's easy to re-flash with solid OpenWrt, and fairly inexpensive: https://openwrt.org/toh/netgear/r7800 )
One thing I found very important was to plot the scatter of PING vs. Download speed (my Upload is super stable).
Then I created the probability density function of that (using KDE) and used it to get a better deal from my network provider - I could prove they were selling me a slower network most of the time - they now allocate 70Mb/s on a 50Mb/s deal so just that they hit the 50Mb/s most of the time.
It also allowed me to clearly identify some outlier situations where my network suddenly became 'bad' and I could show evidence of it being bad over time.
Certainly a project that should see more traction!
If anyone is interested feel free to contact me. Would love to learn about what you would want in such a device.
I don't think regular consumer is their target audience however, they seem more geared to ISP's and government agencies who need performance data over a large amount of connections.
I suppose the problem with wanting to build a small device that does this, is finding the balance between cost-effective and actually being able to pull off 1gbit throughput on it.
Of course, this all assumes you were going the repurposed hardware route as opposed to design from scratch. Doing it from scratch would have it's own costs and pitfalls. (Board design, manufacture, licensing costs, etc)
I ping Google, CloudFlare, Quad9 and Amazon EC2 and plot it to see the any unusual spikes.
Now you need "complicated" things: prometheus, grafana, docker, etc.
I am a bit puzzled. Is it because sysadmin tutorials from 2000's are not showing up anymore in Google ? Why these tools failed to stay popular ?
With Prometheus and Grafana you can monitor an entire data center and add visualizations for metrics as you need them with no change to the monitored hosts.
TLDR: the old software stack could be made to do this, modern monitoring stacks do it out of the box.
All of the SEO skiddies are the ones pimping Kubernetes for your personal homepage and a few links.
And it shows. Lots of ways to build things fast, but nobody's building cool, lasting shit.
No way to make money with something that's well-built with the intention to last a life time. It's the "planned obsolescence" of the software world...
I generally agree with the rest of your point, but the new-fangled infrastructure tools do have some merit and I look forward to seeing where we go with them.
For this kind of setup, prometheus and grafana are not complicated (close to 0 configuration), and docker mostly works out of the box on linux hosts, barring nftables shenanigans. You end up with something that "just works", is pretty, and does not require fiddling.
Technically I do find this overkill for "home network monitoring", but at least it does not require k8s so I won’t complain.
One would think that should be a hell of a low bar. It’s like having to choose a car and going “Trabant will do, at least it does not require a crew of seamen”.
I've been using InfluxDB at home just because of how easy it is to setup and use.
The best I can make of it is to treat docker as a package manager. All the dependencies in one place.
Just works, sits neatly alongside a few extra services (managed via compose) in a little ARM dev board and has kept on ticking for around five years without any hassles using exactly those bits you mentioned.
The new things are shinier, I think, and on the other hand a lot of the old, simpler tools have become less relevant.
As an aside, I recall a younger self annoyed at the inability to get data out of RRDTool into a nice SVG chart. These days I just want the data in whatever format the best tool spits out...
The cron gives you low granularity (and for some things, it may matter to measure things that changes in less than a minute), and flexibility (both in what you can measure and how you can arrange and correlate the different pieces of information).
I worked in FAANGS where simplicity really mattered over hype.
I ran K3s for a few weeks on ODroid hosts at home to get more k8s experience.
And the load of dozens to hundreds of data points in rrd is shockingly large. Our infrastructure utilization dropped by 30% when I switched from a Graphite/collectd setup to gathering MORE data more frequently with telegraf and influxdb.
I'd say it is easier and more powerful than rrd and cacti (used them like 10 years ago!)