NTP Pool Bad Actors: The Rising Sophistication of Network Scanning
netpatterns.blogspot.com
netpatterns.blogspot.com
Shodan.io fancies itself as a search engine where you can search for IOT things (webcams, refrigerators, etc) that are on the internet.
They have no technical issues doing this in the IPV4 space, because it's easy enough to scan every single address in the space.
This isn't practical for IPV6. So, it seems they wanted a way to identify every IOT device in the IPV6 space without having to scan all of it.
At least one approach they found was to join the ntp.org pool[1], and effectively donate server time. Since pool.ntp.org is the default NTP server listed for many linux distros (and thus, probably IOT devices), they now are getting live connections from the exact devices they want to index on their search engine.
Once you connect, they scan you back on 100 ports on so (ports unrelated to NTP) to see if you are a webcam, router, or whatever else they want to put in their index.
Pretty shady. Kind of like volunteering for a charity so that you can raid their internal mailing list for spam purposes.
just wrap ntpd in firejail[1].
ipv6-per-networked-process pretty much brings an end to portscanning.
Granted that means trusting NIST over someone else.
The other options are getting time from a cellular modems, GPS or the most extreme running your own atomic clock.
Sort of, but in the global order of shadiness, this ranks pretty far below what your typical multinational corporation does to track an individual. It's pretty ingenious actually.
It's not like your service has to expose all its secrets to an unauthenticated client sending you a GET / HTTP/1.1.
I hope the NTP pool terms of service will disallow such behaviour in the future, otherwise I'm removing my server from the pool.
If your device has a publicly routable IP address, it should probably be behind a firewall. If it's not behind a firewall, it's going to get scanned. Relying on security through address space size is stupid.
> What was most puzzling was the fact that the devices that were targeted had randomized IPv6 addresses and were not published in DNS or any public record. For all intents and purposes they were hidden safely within my lab network.
That is just wrong, and it's wrong in a very glaring, obvious way: The devices were not "hidden safely within [his] lab network", because he was not using NAT. They were connected to the internet with publicly routable IP addresses, which they were using to communicate with other hosts on the internet.
Edit: That being said, it's a great writeup, and some good technical work was done to figure out what was going on, and I enjoyed reading it. But the starting premise seemed to be "how could this be happening, my devices are hidden!" and I feel like it should have been "oh, it makes sense that someone would do this since my devices are no longer hidden".
But what I was getting at is that however illusory the safety of NAT is, the author seemed to be instinctively assuming that his IPv6 devices had it, because we've spent 20 years training ourselves to assume that everything is behind NAT. It's a new world.
http://www.internetsociety.org/deploy360/blog/2015/02/ipv6-s...
Unfortunately privacy extensions are not enabled by default on most linux distributions and server operating systems.
Oddly after being an unwitting participant in one of the NTP amplification attacks I set up my own stratum 0 NTP server based on the beaglebone black, the adafruit GPS module, and the PPS time keeper code. So all of my machines only talk internally for time, although initial installs still use what ever the distro packed into them. I brought up Debian on a Dragonboard 410c and it sets the time via an NTP call during the initial startup process. (or fails to set the time as NTP is blocked egress/ingress from the firewall)
On subject, I have always though having a master internal ntp under strict controls should be used, so good on you for that, I dont see ntp payed attention to far too often in the enterprise.
1. You have to use some awful hacks if you want to ssh into it (IIRC it ignores ARP requests for some reason) 2. The GPS only works on Android
That said it's still a nice upgrade hardware-wise from something like a Raspberry Pi. The software support is just spotty in a few areas
Surely if all your internal hosts are talking directly to the external NTP servers you are doing it wrong? My gateway box sets itself by pool.ntp.org and the internal ones set themselves by it. I thought that was how you should do things (even if it isn't a rule, it is only polite to try not overuse a public resource).
> These addresses are 128 bits in length
Only 64 are relevant here: once you make outgoing connections from any address a scanner knows there is at least one active host in that /64 and there may be more. Though of course a 64 bit address space is still impractical to scan.
Depends on how many internal hosts you have, and ease of configuring them (there is a DHCP option for ntp server, but not everything will use it), and available hardware to setup ntp servers: ideally you would have each client syncing from three servers so the clients drop a broken server.
For a small network (a home or small office network for instance) you likely have one incoming router anyway, so if that dies clock setting is the least of your worries until it comes back up, meaning setting the internal hosts by that one clock is as fine as anything else.
For a larger network you have redundant everything, including edge servers that can act as your multiple NTP sources for internal hosts rather than every internal host poking the public pool.
I'm not so much worried about the router dieing and losing sync with the outer world; but about the router's clock going wonky: I've seen computers where something got initialized wrong and they were 30 minutes slow after 10 hours (reboot helped in that case, but sometimes it's the oscillator is just too far off)
It is unlikely that Shodan is going to hack your Raspberry Pi specifically because then people will go after them. But hackers of many hat colors will use the freely available information it gathers for their own purposes, so act with this in mind.
If your server isn't meant to be contacted by the world, then put it behind a firewall. Don't count on an obscure address to keep you hidden.
For people who do care about network security, it is a very interesting post for a few reasons.
If someone is harvesting IPs from NTP queries sent to Debian infrastructure for intelligence gathering, that in itself is a big deal.
It is also a somewhat novel technique - if an attacker is placed such that they can see the first NTP queries sent by a new install, they are well placed to target that device before it is fully hardened/patched, because they likely see some of the first packets sent by that install. It is really rather clever.
It also points to the sort of correlation we need to be getting better at - orchestration isn't just for devops weenies anymore, and this sort of thing is only going to become more sophisticated.
I've been noodling about with making a tool for making this sort of analysis easier, but there are a number of problems to fix, not least of which is the sheer volume of data generated watching the wire, even on my home network (which is a bit absurd for your average home, but tiny compared to a business of any size).
Debian doesn't run the NTP pool.
[0]: http://lists.ntp.org/pipermail/pool/2016-January/007758.html
I'd think this is something that most network designers/engineers would get. That said, I'd be a rich man if I had a nickel for every time I saw NTP misconfigured.
Using a broadcast time source (CDMA/GSM, GPS/Galileo/GLONASS, WWV/WWVH/international equivilents?, ATSC broadcasts, etc), or choosing a trusted time server would be alternatives; you could also use a telephone time service and leak your phone number instead of your ip address.
And the fact that they sent packets back to you (after you sent them packets) is not surprising either. However, if you can show that a full-blown port scan occurred after you sent them packets, then that would be odd. I did not see evidence of that in the article... did I miss that?
"It takes less than five seconds for your address to be harvested and scanned. The entire scan takes less than one second and scans over 100 common TCP and UDP ports. "
The mantra of the new Internet is "Trust no one". :(
And even if I do (I might trust them more than a more random selection of individuals, taking them doing something for the community as a point in favour of their character) that wouldn't stretch to trusting they were not unknowingly compromised by a third party.
You also hope that visiting a web site doesn't lead to the web site running a scan on your IP address. Or doing any number of other nefarious things.
This kind of trust is at least most peoples assumptions. If it weren't, there would be no outrage or even surprise at the nefarious behaviors which abuse our trust.
a) I never trusted them, so this is fine and to be expected
b) I sort of trusted them, so this is surprising and not polite.
I lean towards (b).
The normal ipv4 methods of "scan the entire address space" fail in an ipv6 world, so shodan needs some way of mass havresting ips for them to gather scan results for...