Here’s what you find when you scan the entire Internet in an hour | Hacker News Reader

Here’s what you find when you scan the entire Internet in an hour

washingtonpost.com

105 pointsanxiouser40 comments

People willing to exploit insecure sites may be able to scan faster - legality is a different issue.

http://census2012.sourceforge.net/paper.html

Starting with one device and assuming a scan speed of ten IP addresses per second, it [the scanner] should find the next open device within one hour. The scan rate would be doubled if we deployed a scanner to the newly found device. ... We did this in the least invasive way possible ....

I wonder if/when attaching a widely accessible and easily exploitable device will be considered illegal (attractive nuisance, negligence, public nuisance, contribution to a crime)?

To leap to a car analogy, if a driver leaves the keys in a vehicle ignition, and the vehicle is stolen and used to commit some other crime, does the driver face criminal penalties or civil liability?

Should a computer vendor or user who neglects to secure their systems or network face penalties or liability? Should external entities do wide scans to encourage better security? I think that a "name and shame" approach aimed at vendors who ship or install insecure-by-default systems could be effective.

You should not be criminally liable for something that was merely caused by your actions.

http://en.wikipedia.org/wiki/Strict_liability_%28criminal%29

Thread from yesterday linking to the actual lib: https://news.ycombinator.com/item?id=6226105

reading the article, it might seem that it's some sort of futuristic technology, but it's been used since 2002 (scanrand)

the downside of stateless portscanning is that you are trading speed for false negatives.

the downside of stateless portscanning is that you are trading speed for false negatives

That's interesting. Could you explain this in a bit more detail? Unless I missed it, the thread from yesterday didn't discuss this.

Since it's stateless, all info is encoded in the outgoing packet. If the outgoing packet (or the reply to it) is lost, it will look exactly the same as if the server didn't respond - after all, the scanning tool has no local state, and thus can't track if an address has been pinged/re-ping it. The port map is entirely drawn based on incoming packets.

I don't know how they progress through the IP space, but couldn't they simply solve this by doing it in a deterministic manner? At progress N they should easily be able to tell that A has been scanned. Iterating three times through the IP space all IPs that haven't answered should have gotten the connection attempts.

That might introduce the same overhead that maintaining state does in the first place. It sounds like they're sending out at least a million requests per second.

Thanks! I get it now. Good explanation.

Probably why it's used as a statistical tool rather than a security tool.

This can't be stressed enough. It's not all about bandwidth, packet loss is very real.

I'd be interested to see how these, apparently rather frequent, port scanning exercises are being factored into 'attempted cyperattack' statistics.

New scanning technique shows sharp increase in internet scans!

This looks like the code for the project

   https://github.com/zmap/zmap

reminds me of dscan, originally from about 2003 or so, which itself was built around the time of scanrand.

https://github.com/dugsong/dscan

> ZMap is capable of performing a complete scan of the IPv4 address space in under 45 minutes, approaching the theoretical limit of gigabit Ethernet.

Are they scanning all ports, or a subset, or just one?

Just one in the 45 minutes quoted. It's still impressive.

There's no timezone on the Time of Day chart. Any good guesses?

Eastern time. From the article:

In any event, the best time to scan the Internet, at least from Michigan, seems to be early in the morning.

internet != the web

Yes... They are using the term 'internet' here, and only use 'web' when talking about HTTPS. They are port scanning hosts, at the TCP or UDP level. That seems correct to me?

What's the distinction in this context?

The article's title is misleading. It speaks about the web mostly, not the internet (wich you can't scan in an hour btw).

You can. You can send an IP packed to every host in the internet and hopefully recieve a reply. That us the internet scanning.

I think he meant scanning all ports, UDP+TCP+ICMP etc etc

That's meaningless. That's like claiming you didn't really visit a country until you looked under every trash can.

There is a sweet spot between looking in every trash can and visiting only one of the biggest cities :)

I'm not sure there is. I'm not sure one can be truly sure he scanned the Internet before impersonating every host. Can't know anything before trying out the inside of every skin.

After all, what would you know, as a traveller, about simple lives of local people?

I've spent one hour of my life in Germany, when I was 11 years old, in a transit lounge in Frankfurt. I have 'visited Germany', but not in any real sense.

Everybody have their own threshold. I only consider city visited after I spend a night there.

There's more to the internet than just port 80, so to declare that a scan encompassing only a single port on each host is a scan of "the entire internet" is somewhat mistaken.

The more correct title would be, "a scan of the entire World Wide Web."

Even that's not correct though. Port 80 is just the default port. Not to mention the number of web servers only doing HTTPS on port 443 and not 80.

More correct would be "A scan of world wide web servers running on the default port 80"

https://zmap.io/paper.pdf From page 14, Section 8, titled, "Conclusion"

"We experimentally showed that ZMap is capable of scanning the public IPv4 address space on a single port in under 45 minutes, at 97% of the theoretical maximum speed for gigabit Ethernet and with an estimated 98% coverage of publicly available hosts."

I doubt they used the same port in every scan.

https://news.ycombinator.com/item?id=6234877

Why be in doubt when the research is published?

"Single port number" doesn't mean "same port number every time".

So what does "scanning the public IPv4 address space on a single port in under 45 minutes" really mean then?

Did you even read the documents?

They visit one port in the whole internet. This doesn't prevent them from visiting another port of the whole internet next time.

This makes "they visited the whole internet" true, "they aren't limited to web only" also true.

I realise that my comment was not so clear, sorry about that. Yes, to me scanning the whole internet means at least the full port range in TCP (and why not UDP too).

My 'rant' is really about the article sensational title promising to let you know about the result of scanning the entire internet really fast... wich turns out to be about scanning web services. The data is however interessting.

In other words,

  ((16.8 + 16.8) * 1e6) * (2^15 + 2^14 - 1)) / (24 * 60 * 60)

  ((IANA + RIR address) * millions) * (registerable port range)) / (day-seconds)

  19 million scans / second

Reply on news.ycombinator.com