Support Diary: The LAN of 16M Hosts

I had a $cow-orker who once did the exact same thing -- configured an interface to use a /8 subnet mask -- except it was on the new Cisco ASA firewall we were deploying for one of our customers (a government water utility) and it wasn't a stupid default; he explicitly configured the wrong subnet mask and told the ASA that there were ~16.7 million other devices directly connected to that particular interface!

Not surprisingly, they experienced various problems over the next few weeks due to "the firewall" -- including breaking the application they used to provide their vendor remote access for monitoring and maintenance! That had the biggest impact but there were also various web sites they couldn't access, they lost the ability to send and receive e-mails from some other organizations they worked closely with, and so on. Oh, and either Pandora or Spotify (I don't remember which) also stopped working -- she could deal with some of the other stuff but the manager of the organization was NOT happy about that one!

Eventually, a few weeks after the firewall replacement, I was asked to go investigate the issues they had been experiencing since the new firewall had been installed, quickly noticed the incorrect configuration on the interface, and became the hero to everyone (well, except for my $boss, who had to reimburse them for ~27 hours (IIRC) of on-site time, but that's another story).

Subnet masks matter!

Totally unrelated to the actual post and apologies for what is arguably spam, but I would love to give a shout-out to prgmr for being the best VPS hosting company I've ever used. I've been with them since early 2011 and run all my most critical personal services (email, static web, etc) on prgmr machines. I particularly enjoy how VPS management (including OOB!) is done through SSH with a pubkey you supply on the billing site: https://i.imgur.com/ckRJzDW.png

I don't record any reliability stats, but I can't remember a time one of my VPSes was down without being announced first. Reboots and downtime do happen because they are extremely on top of Xen security patches, hardware maintenance, and everything else. My favorite prgmr email was last year when a scheduled maintenance got postponed by two days only six hours before it was planned and they sent me an email apologizing that my machine wasn't down just in case I had already planned around it: https://i.imgur.com/yusTBPG.png

Full disclosure: one (only one!) of my prgmr VPSes is billed to me at $0/month because I helped get FreeBSD Xen/PV running at prgmr way back in the FreeBSD 9.0 days when FreeBSD Xen/HVM was rather unusable. These days PV isn't even an option (afaict) for prgmr's VPS and the newest HVM-mode FreeBSD is available straight from them in the management console as a first-class OS alongside CentOS and friends. If that arrangement ever ends I will immediately switch to paying for the same machine and wouldn't even consider shopping around first :)

I find it a bit surprising that classful networks is not common knowledge. I guess it's just a sign that times have changed. This was a basic thing I learn when I first learned about subnets.

TL;DR - we'd like someone to add warnings to ifconfig etc. if no netmask is supplied. I could do it but don't have the time.