Masscan: Scan the entire Internet in under 5 minutes | Hacker News Reader

Masscan: Scan the entire Internet in under 5 minutes

github.com

147 pointspmoriarty29 comments

Note: your ISP and third-parties probably won't like this very much.

There's a great talk from Defcon 22 on using Massscan for security research:

https://www.youtube.com/watch?v=UOWexFaRylM

This is a hell of a piece of engineering. Really fun to read the README, a custom TCP/IP stack is genius.

Indeed. I was quite impressed by his solution for randomly iterating through the IP space. I've had use cases before for randomly iterating through a space while ensuring to hit every space and they've never been quite as efficient in space/time complexity as his.

[https://github.com/robertdavidgraham/masscan#randomization]

can you elaborate on what/how one would randomly iterate a space? i imagine it's like trying to draw a space filling curve on the place (for spaces with 2 components, such as a coordinate)

To avoid caching/clustering/grouping effects.

My terminology was off, I more-so meant range rather than space.

"It's the program that scanned the Internet in less than twelve parsecs."

From the README:

"Note that it'll only melt your own network. It randomizes the target IP addresses so that it shouldn't overwhelm any distant network."

Title should be updated to include that this system scans only via IPv4. Doing such a thing with IPv6 would be a little more surprising. (7.9228163e+28 times more difficult)

Surely not all IPv6 addresses have been allocated yet though? Most of the web is still using IPv4.

I have a /64 allocation, glhf.

There is a false assumption that IPv6 will make mass scanning like this impossible. In reality you just need to be more clever about it. (Remember way back when people used "needle in a haystack" security for dial-up systems, because nobody would ever have the resources to call every phone number in an area code?)

Link-local multicast (the replacement for ARP) allows tools like alive6 to very easily enumerate all live v6 addresses on a network. So once a spear phishing attack is sucessful, you can still scan the entire internal network.

Google hacks like "site:ipv6.*" and passive DNS monitoring allow you to easily separate used vs allocated/announced subnets on remote networks. IPv6 breaks in strange ways when you firewall ICMPv6, so ping scanning a subnet has become much easier.

There was also a great talk (i'll try to dig it up) that talked about predictable patterns in DHCPv6 implementations, so you can cut down v6 to a near v4 search space.

The best part of all is that very few security products on the market really support IPv6 correctly, so I suspect we will see more advanced attacks being possible because of IPv6 in the coming years than things being stopped.

>There is a false assumption that IPv6 will make mass scanning like this impossible.

Well, this is an IPv4 brute force search, so technically a IPv6 brute force search is still impossible.

You are correct though, no one is going scan something that is 99% empty by brute force.

Here's [1] an example of using Masscan to scan the IPv4 space for shellshock.

[1] http://blog.erratasec.com/2014/09/bash-shellshock-scan-of-in...

I might have missed it while reading the README, but can someone ELI5 why do we need to randomize our scans?

Can't we just go scan one-after-another IP address? Is this because such scan can easily be detected by ISP?

Think of the internet like a tree, where the root is you and all other IPs are the leaves at the end. IPs close together tend to share more path of the tree as you attempt to reach them from the root. If you are sending an overwhelming amount of packets in one direction for too long, you have a higher chance of harming nodes (i.e. routers) along that path. Randomizing your end goal on the tree, by definition, equally spreads the packet spray accross the tree.

This is how they can claim: "... it'll only melt your own network. It randomizes the target IP addresses so that it shouldn't overwhelm any distant network."

Note "shouldn't", this was probably added due, in part, through use-case. If I were not scanning the whole Internet, and instead just scan a small section. Masscan has less of a space to randomize through, which means the tree is smaller and the shared paths are more frequent.

How likely is this to be used for anti-piracy efforts? I don't hear much about en masse copyright enforcement these days, but it seems like the ability to quickly scan large IP ranges would allow one to periodically (every couple minutes or so) obtain a list of every single seeded file in the US, at least for the people not using a VPN.

That's not how torrents work. You can't connect to a port on a seeder and get a list of torrents it's seeding. Even for trackers, you have to request a specific URL to get a (partial) list of available seeders.

Oh I see, please forgive my ignorance of the torrent protocol. I was under the (apparently mistaken) impression that, like Napster, one could enumerate a user's shared files simply by knowing their IP.

On the other hand, it is easy to crawl DHT. "Crawling BitTorrent DHTs for Fun and Profit" (2010) says "We find that we can establish a search engine with over one million torrents in under two hours using a single desktop PC".

Sorry if I seem naive but, is this even legal? ...

I am not a lawyer, but as long as your local laws do not prohibit you from pinging a given IPv4 address, then I can't imagine any issues. Being technically legal doens't mean you won't step on some toes though. Everyone who runs full internet scans has reported getting lots of exclusion requests, (baseless) legal threats, and even retaliatory DDoS attacks coming back at their source IPs.

Massscan ships with an exclude list which you would do well to utilize:

https://github.com/robertdavidgraham/masscan/blob/master/dat...

If you try to run an internet wide massscan without this list, it will stop you and give you a warning about how to use the list. You then either manually override the warning, or use the list.

Some interesting emails are transcribed in the comments of that exclusion list. I liked the one from General Dynamics.

Distributing the software is probably illegal under the Computer Misuse Act here in the UK.