HardCIDR will query ARIN and a pool of BGP route servers | Hacker News Reader

HardCIDR will query ARIN and a pool of BGP route servers

github.com

85 pointscrystalPalace47 comments

Beware, this script will hose / clobber and then silently clean up (delete) a ton of various different files if files with those names happen to already exist. To see the exact file names you'll have to carefully pick through the script. So obviously? run it in its own directory, which is no guarantee of safety, but should be safer.

If you have a subdirectory where you run it named after your email hostname (such as "example/" for "example.com"), then it will prompt you to "overwrite the contents of the directory" and then, if you accept, it will not only overwrite the contents, it will remove the entire contents with:

    cd $outdir
    rm * 2>/dev/null

There's a slight violation of user expectations here. Removing and replacing the contents isn't quite the same as overwriting the contents. It may be a fine line, but it's better to err on the side of protecting the user's files, not deleting them, when deciding where to come down on that fine line.

And if $outdir is empty or not there, it tries to detect that by first doing a check for -d $outdir, but this won't save the user if $outdir gets moved aside by another process while they are reading the prompt and before the cd happens, leaving them in another directory. Hopefully the user has rm aliased to rm -i but that still won't help since the rm is being run in its own shell in the script.

I know we're not supposed to focus on the negative here on HN. I'm sure the script is awesome for whatever it does. Just be careful out there!

That is bad. One should always use set -u on scripts that rm any files or perform any destructive actions. One can always unset that if you need to accept args further along in a script to use in a variable.

You seem to have a good eye for picking up problems in scripts, and enough of an interest to write about it. Have you considered submitting a PR with your suggested changes, or if you don't have time for that, perhaps an issue with your problems?

No, because the README for this project leaves me having utterly no idea who the audience is, what it is for, why it exists, etc., so I wouldn't feel I have the right background to do a proper PR against it.

I wish modern OSes made this easier. I would love to have an easy bullrtproof way of saying "give me a temp directory for writing, don't let me write anywhere else, clean up my directory after me".

Chances are that every system this script runs on has "mktemp," and trapping exit makes it easy to clean up when things are finished.

https://www.mktemp.org/manual.html

http://redsymbol.net/articles/bash-exit-traps/

Yep, makes it much simpler to write scripts like this https://github.com/psypete/public-bin/blob/public-bin/src/st... (run an application that's stored in git, but in a temp working directory, and clean up after)

But, how can I make super sure some combination of my mistakes, undefined variables, failed cd commands, race conditions, etc, can't lead to me clearing the wrong directory.

I'm sure this is excessive paranoia, but I have never been confident to release a script that classes a directory, I'm to worked about things like subtle differences in how mktemp works on linux and mac for example.

on linux the following might work:

    - enter a user and mount namespace
    - mount a tmpfs, e.g. over /tmp
    - remount everything else as readonly

alternatively

    firejail --overlay-tmpfs <command>

A good usecase for Docker, maybe? Put the script in a container and let it run wild, then delete the container when it's done.

OK, but what does it do? The README is pretty sparse. Some examples would really help.

Edit: the header from the script is good, toss it into the README for great success.

It simplifies the process of finding the IP address blocks allocated/assigned to an organization.

> HardCidr is written by Jason Ashton, Senior Security Consultant at TrustedSec

I'm guessing it was written with pen-testing in mind.

This might give it some more context: https://www.trustedsec.com/march-2017/classy-inter-domain-ro...

It all makes sense now! This is much more pertinent than the code if you don't know much about the subject matter.

Note that this script installs "ipcalc" (or really, whatever is in http://jodies.de/ipcalc-archive/ipcalc-0.41.tar.gz) without user interaction.

I'm generally pretty not okay with scripts that curl | tar things (or apt-get install things, which this does if it's run on a linux) from the interwebs without my explicit consent.

That shouldn't be an issue if you don't run it as root.

By running it as root, I'd argue that you did give explicit consent for the script to do anything it wants.

Sorry, no. The opposite, in fact. A script that demands to run as root on my computer needs to be extremely well mannered.

I certainly don't disagree with that.

If one downloads and blindly runs some random script as root, however, you are effectively allowing it to do anything it wants.

It sounds like javajosh took the time to look the script over first which, of course, is exactly what one should do.

Oh, this is superfly. Easy way to build your own up-to-date ASN DB, similar to the one from Maxmind. Think: embellishing Apache/Nginx logs with up-to-date information about the IP address of the client, including ASN/OrgId. Useful for identifying snowshoers spreading their footprint across a lot of discontiguous IP addresses in one ASN/Org.

Route server operators would prefer people didn't do something like this against their devices.

Agreed, that's why I'd run it against systems I operate that have at least one full view of the DFZ.

Oh, if that's what you want:

https://iptoasn.com/ https://pypi.python.org/pypi/pyasn

I've been working on tweaking pyasn a bit and building a service around that.

Or if an API works for you, https://ipinfo.io

    $ curl ipinfo.io/8.8.8.8
    {
      "ip": "8.8.8.8",
      "hostname": "google-public-dns-a.google.com",
      "city": "Mountain View",
      "region": "California",
      "country": "US",
      "loc": "37.3860,-122.0840",
      "org": "AS15169 Google Inc.",
      "postal": "94035",
      "phone": "650"
    }

    $ curl ipinfo.io/8.8.8.8/org
    AS15169 Google Inc.

And ASN details are available on the web, eg https://ipinfo.io/AS15169

I'd like this service a lot more if it had a "last_refreshed" field. BGP/network-announcement hijacks aren't exactly common, but it'd be a useful bit of info to have in terms of determining how reliable the announcement is.

By "reliable" it sounds like you mean "legitimate"? As in, $asn is "authorized" to announce $prefix? "originAS" exists for that purpose.

Also, it's not clear what they're (ipinfo.io) using as their source for the ASN. Are they simply reporting the ASN as provided by ARIN, etc., or are they actually running BGP and reporting the origin ASN as they see it in announcements. My money would be on the former, in which case any prefix hijacking would not affect the data reported by ipinfo.io.

By "reliable", what I mean is that the IP address to ASN mapping is as up-to-date as possible.

I don't think a "last_refreshed" date would be that helpful, though. Netblocks aren't being shuffled around very often. I just looked at a previous employer's assignments and it was almost a decade ago that it was last updated. It's still 100% accurate, however.

Regardless, if you want it, that data is available from the RIRs. Go crazy.

That works for one lookup. The service I'm building on top of pyasn uses zeromq and can do 100,000+ lookups/sec

There's often a disconnect between something like that which works for one address, and something I could actually use to do bulk lookups on 5,000,000 addresses to generate reports.

Is this how Maxmind gets their data?

If you just want to build your own IP-to-ASN table, you can download dumps of "RIS Raw Data" [0] from RIPE and parse them if you don't yourself run BGP.

I'm a network engineer at an ISP and it's pretty common to use something like this for analyzing traffic network when considering peering sessions, for example. Even if you don't run BGP, you could use it for answering questions like "how much traffic do we send to/receive from Facebook?" and such.

RIPE's RIS dumps are performed every five minutes from more than a dozen different "vantage points" across the Internet.

ARIN used to provide an "originAS" file [1] but it looks like they quit doing that a few years ago. You may be able to find some interesting stuff browsing around /pub on their FTP server, though [2].

[0]: https://www.ripe.net/analyse/internet-measurements/routing-i...

[1]: ftp://ftp.arin.net/pub/originAS/

[2]: ftp://ftp.arin.net/pub/

I always forget that this exists - thanks for the reminder.

I guess this is on hn because it sounds cool? It's impossible to understand what it is.

From the top of the script:

> A tool to enumerate CIDRs by querying RIRs & BGP ASN prefix lookups

> Currently queries: ARIN, RIPE NCC, APNIC, AfriNIC, LACNIC

> Queries are made for the Org name, network handles, org handles, customer handles,

> BGP prefixes, PoCs with target email domain, and 'notify' email address - used by

> some RIRs.

> Note that severl RIRs currently limit query results to 256 or less, so large

> target orgs may not return all results.

> LACNIC only allows query of ASN or IP address bloks & cannot search for Org names

> directly. The entire DB as been downloaded to a separate file for queries to this RIR.

> The file will be periodically updated to maintain accurate information.

> Output saved to two csv files - one for org & one for PoCs

> A txt file is also output with a full list of enumerated CIDRs

> Author: Jason Ashton (@ninewires)

> Created: 09/19/2016

This might well be Arabic. I have been in the industry for over 10 years and that explanation is meaningless

These are all Internet Protocol routing features. "CIDR" refers to classless interdomain routing, iirc.

> ARIN, RIPE NCC, APNIC, AfriNIC, LACNIC

These are the global agencies that manage the IP address allocations

you might want to go back an re-read your networking books.

Care to point to a book that talks about these acronyms. They are not in comer or Stevens both which are networking bibles

Ah, but far from the only networking bible. You seem to have forgotten Halabi, Perlman, and a raft of others that discuss (at length) routing and the public Internet.

Some of these acronyms are specific to BGP. You could work in networking for years and not encounter some of them, especially if you aren't running BGP.

As far as "bibles" go, however, Halabi's _Internet Routing Architectures_ is the BGP variant.

TCP/IP Illustrated might not mention CIDR since it was still pretty new when those books were written. My copies haven't been opened in years so I can't be sure.

If you've performed any subnetting in the last 15 years or so, however, I fully expect that you have encountered CIDR.

That's because "ARIN, RIPE NCC, APNIC, AfriNIC, LACNIC" are names of organizations, not technical terms.

The very first line has enough information to know what it does:

"A tool to enumerate CIDRs by querying RIRs & BGP ASN prefix lookups"

In other words, it queries two sources (regional Internet registries -- the organizations referred to above -- and information from the BGP protocol) to enumerate blocks of IP addresses.

On the other hand, it's not meaningless for those of us here who are network engineers.

Just because you're "in the industry" doesn't mean you know everything about everything. I'm not a developer so half the things discussed here on HN are waaaaay over my head.

I don't think the industry you've been in for 10 years is the same one that is relevant to this post and explanation.

Those are some cool acronyms that I've never heard of. Reading the README does not explain any more. It's quite the mystery how this got to the top-30...

Perhaps because some of the people here do know what the acronyms stand for?

I can't find how to make it work for european companies. For instance, fnac.com doesn't give any result with the -r option. Did I miss something ?

>The script with no specified options will query ARIN and a pool of BGP route servers.

To what end?