API providing threat analysis of any given IP address | Hacker News Reader

API providing threat analysis of any given IP address

fraudguard.io

121 pointsdontbesalty77 comments

I operate two non-exit tor relays. They have both return risk_level: 4 with the "threat" being "tor_tracker". What threat is posed by a non-exit tor relay? What does the "tracker" part of "tor_tracker" mean?

Exit node: https://atlas.torproject.org/#details/463DC28452F676B7A6597A... https://fraudguard.io/?ip=5.196.143.10

Non-exit node: https://atlas.torproject.org/#details/EF4BD6E8E5817690B79C67... https://fraudguard.io/?ip=108.61.199.202

It does not seem to make a distinction between exit and relay nodes, they are both deemed "tor_tracker".

A number of large vendors like Barracuda and Tipping Point do this. A friend of mine one time found that they were unable to log in to pay their TV bill a few days after setting up a relay node. After he made enough noise about this, their engineers investigated, and sure enough, Tipping Point flags a relay node as suspicious for no good reason, and the IPS was blocking his traffic.

On a side note, companies really don't like to deal with people who can't access their site. Being told over and over "It's your ISP", even when on a conference call with said ISP (and nevermind how difficult it was to get that set up with someone who understands the problem) was infuriating.

Not sure, but I've been put on an IP banlist for simply operating a non-exit relay in the past. Had to switch my home IP. The myth that is perpetuated that if you run a non-exit relay that you'll have no problems with IP bans is definitely wrong.

It's my fault. I'll try to find a way to get this data and eventually adjust my code to not include non-exit relays or at least recategorize them as a lower severity.

Not your fault - not talking about your service specifically! My IP was banned by several other companies (I assume from a list that was purchased from a third party) like Hulu/Netflix because I was simply relaying non-exit traffic.

TOR and people who speak for the service often say that it's safe to run a non-exit relay. It isn't. It's tracked and punished. I know from first-hand experience.

I've been running Tor relays from home for about a year. Only sites I've found to block me are Monoprice and Apple's support forums. Hulu and Netflix are fine.

Does Tor use it's own port? If so, how hard would it be to switch it to use something like 80, 23, or even 8080?

The IP of a Tor relay is publicly distributed, that's how other nodes know to connect to it.

You can see for yourself: https://atlas.torproject.org

Although at a guess, most of these services probably do the simplest thing possible and go by an open port.

Why would they increase the threat profile of an IP in any way at all? An exit node, sure, but a relay? What possible threat could a relay pose?

Agreed it was a mistake on my part, I'll fix it up shortly to include only exit nodes.

Thanks, I really appreciate it.

Due to a similar IP reputation service, I couldn't pay my taxes from home this year, just because I had run a non-exit Tor relay recently. It's a big problem.

ETA: I'd also strongly recommend not marking an IP address specially in any way just for having a non-exit relay -- from what I've seen, the clients of IP reputation vendors also don't understand the distinction, so they block both kinds if told about both kinds. It's an attractive nuisance.

> Due to a similar IP reputation service, I couldn't pay my taxes from home this year, just because I had run a non-exit Tor relay recently. It's a big problem.

Something doesn't seem correct to me about this...

Doesn't really matter - it happens. I'm not specifically talking about the OP's service but rather another third party list that was purchased by other companies. My IP was banned. Whether it should have been or not is up for debate, but I can tell you I was confused for a week straight when I was getting cryptic Hulu/Netflix/Bank/etc error messages.

i worked in fraud analytics and the consensus was that fraudulent activity weighted higher from certain networks than from others. i wasn't in the position to question the decision regarding threats where i was, i just implemented the risk level rating in software.

Most of this data is available for free download via an organization called FireHOL. See http://iplists.firehol.org

It's not clear what this adds on top of FireHOL.

If the creators are on HN, curious to know what sets this apart from other threat intel services like IBM X-Force, ThreatConnect, VirusTotal, Carbon Black, etc.

I'm Ryan, one of three devs that built FraudGuard.io. The answer is basically price. We do EVERYTHING ourselves and try to keep price extremely fair. We have plans that start at $10 /month and we also have a small free-for-life plan too.

Thats the best answer. The secondary part is we want a simple way for any dev off the street to integrate with api.fraudguard.io in 5 minutes. Imagine you are a small company and you want to limit back office access to you CMS to only IPs outside of Germany. That takes just a couple minutes. Or you want to allow back office access to you CMS from everywhere but you want to decrease the session timeout to only 30 minutes if the originating IP is a tor node or public proxy, etc.

We just got out of beta (literally last night) but so far thats the majority of use cases that we've seen so far.

Hey Ryan, I applaud the free tier and think it's important, but I'd caution that it can be awfully hard to make a business-sustaining revenue on $10/mo. I think it's smart to start low and compete on price, but I think there's value here, it's why there are larger competitors in the space, and if you can climb that value chain and raise prices along the way you will be better off for it.

From a merchandising POV, you could consider moving up-market by building new more powerful features and giving them only to the $25 plan. Then, rename the plans to match who you expect to buy them. People identify with who they are more naturally than they do a number of requests per month.

Also, congrats on shipping.

Edit: Also, pre-fill the form with the visitors own IP -- or better yet just display the results?

Ya this is really great info. Thanks for the feedback. We are still looking at pricing as this is only day one but we're on it.

This one I'm moving up the list - I like the idea "better yet just display the results of visitors IP"

ipinfo.io do this well

And AbuseIPDB

So in the past we talked about this kind of integration. Our only concern would be user generated content. Our users can 100% trust that if an IP is in our system and logged as a level 5 risk level, that that the originating IP hacked one of our nodes and was caught in the process.

Obviously we are concerned that a user that doesn't like (example IP) 4.2.2.1 for whatever reason might create 10 accounts and log this IP as malicious so we would need to tag it as user-generated before we turn it on. But we are considering it

I like this, but I have to tell you what I've been looking for in one of these services for forever.

I help develop a fairly popular webgame. One of our biggest headaches is people who are evading bans by using VPNs (public or not), VPSes, etc. Although we've outright blocked some large chunks of IPs (AWS, for instance), I've never seen a good service that identifies those specific blocks. Sometimes I go manually digging in the case of serious ban evaders, looking up the owners of specific IP blocks, but boy it'd be convenient if there was a service out there that did that.

> One of our biggest headaches is people who are evading bans by using VPNs (public or not), VPSes, etc. Although we've outright blocked some large chunks of IPs (AWS, for instance)

Please don't do this. It's perfectly legitimate to route one's traffic through other nodes one owns.

Please consider other ways of dealing with banned players — perhaps make creating an account slow and/or costly.

We've examined all the options. We already use browser fingerprinting, and that takes care of a good percentage of it, but for the truly committed there are really only two options: Blocking all VPNs, or using supercookies. I'm actually a bigger fan of the supercookie solution, but one of the other developers is staunchly against that. It's an ongoing battle.

The problem with the slow/costly account thing is that one of the big draws of our game is that there's no registration necessary. You can jump in a game instantly by pressing 'Play Now', and you just get named 'Some Ball 1/2/3/4/etc' and tossed with the registered players.

Been there, tried supercookies, found that the same users who are motivated and savvy enough to dodge browser fingerprinting are also able to dodge supercookies.

What do the evil Some Balls do that you want to avoid? I've never noticed an bad behaviour tbh (other than just being not very good).

Tagpro is awesome btw.

Mostly really nasty chat and working against own team. You don't see too much of it nowadays as we're very proactive about that, and have blocked all the major slurs from being typable.

Always glad to find a player in the wild! We're working hard on Next and hope to have the beta open soon.

While I don't believe this type of thinking about IP addresses represents a sustainable approach to an open internet, https://www.maxmind.com/en/geoip2-anonymous-ip-database and/or https://www.maxmind.com/en/proxy-detection-service may be what you are looking for.

Should be possible using the ASN id of the IP address, and I think Domaintools provides this information and some other fields as well, although their API is not free. That won't help with most VPNs though as they just rent servers from various providers, so you need some kind of active monitoring, which might be tricky to implement from a technical and legal point of view.

If you have a BGP feed, you can use that to go from IP -> ASN, otherwise, there are public bgp dumps [1]. Many networks renting servers are pretty simple to flag this way.

[1] One source is http://www.routeviews.org/

Thanks for pointing this out, I couldn't find a way to get the ASN for free, so this is a great resource!

With Shodan you could export a list of IPs that are currently operating a VPN service:

https://www.shodan.io/search?query=port%3A500%2C4500+vpn

Or you could lookup the IP of the user on Shodan and check whether that IP is running a VPN service. Per IP lookups are free on Shodan and it's fairly simple. For example, this is how you'd do it in Python:

    def is_vpn(user_ip):

        import shodan

        api = shodan.Shodan("API key")

        host = api.host(user_ip)

        for banner in host['data']:

            if banner['port'] in [500, 4500]:

                return True

        return False

Here's an overview of the VPN services that are currently on Shodan:

https://www.shodan.io/report/2aLrR79C

This is brilliant. Thank you so much!

Edit: I made a modification for conciseness' sake, and to include SSH in the mix as that can be used as a poor man's VPN.

    def is_vpn(self, ip):
        api = shodan.Shodan(self.config['keys']['shodan'])
        host = api.host(ip)
        return any(banner['port'] in [22, 80, 500, 4500] for banner in host['data'])

I realized it only afterwards but you can actually just look at the host['ports'] property which contains a list of ports that were found open. And you might also want to include PPTP on port 1723.

The major providers do this:

https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.... https://www.microsoft.com/en-us/download/details.aspx?id=416...

Yup, and I seriously appreciate those that do that, but it's the small providers which inevitably end up screwing us over. Some small VPS company has a vulnerable server that someone makes a VPN on, and suddenly we get a wave of ban evaders. It ends up being a constant headache.

If it's default ports, you can just probe on login and deny it. We do this for a game I've admined for, and it's in the terms of service. Common L2TP, PPTP, etc.

Interesting. So if I put a terms of service discussing it on my blog can I nmap all my visitors?

Scanning the defaults for open proxies with a note in the ToS is not super uncommon with some types of services. I don't recommend sweeping random visitors to your website.

Since the OP said:

> I help develop a fairly popular webgame.

... unless the web-based game relies on a plugin that can skirt the browser's sandbox, there's no way to probe for active ports.

You can ask the server to do it for you.

Isn't this a lost battle? Only remotely possible due to IPv4 being so limited?

Do you allow normal users to get around banned IPs by buying a pass or something? (Then you can ban the pass, like 4chan.)

It has been added to Trello

Where do you get your data for botnet compromised IPs?

"Our Team of Engineers track public IPs across a wide scope of popular botnet networks."

How do you track botnets (along with the other collection you do) with a team of three? How many botnets are being tracked and which malware families? This is an especially dubious claim when coupled with another statement you made: "We do not rely on external sources at this time".

I bet he is using maltrail and firehol.

We do use Maltrail along with a whole lot else. You don't have experience with it perhaps? Are you currently employed or angry at your employer?

:-) I built my own API and it is automatically updated every 2 hours. I'm using threatminer to do cross-validation on sampled values.

Just had fed a bunch of IPs from one of my server's auth.log and got the same:

  "country": "China",
  "threat": "unknown",
  "risk_level": "1"

I'm not surprised but with increasing China's activity on the scene we have to do something with it.

I plugged in a few IPs from stopforumspam.com and they all had the same "risk level" as my own IP: 1.

I'm Ryan, one of three devs that built FraudGuard.io. Honestly we have a lot more work to do specifically in spam. With that being said, spam is the least requested collector so far by our users. Just to share in beta we asked some of our heavy users and about 90% of our users preferred our focus was on honeypot collection, spam was less than 2%.

So heres how it works now. We do not rely on external sources at this time. The reason why, because our traffic is so high that no external source at least that we've found will serve our users traffic.

For example stopforumspam.com limits API requests to 20,000 per day. I haven't checked our stats today but during our beta (which ended yesterday) we served more than 20,000 API requests per hour. So even with huge cache durations set its very hard to rely on outside sources so instead we run all our own spam collectors, using our own domains, etc.

Correction, API providing:

502 Bad Gateway

nginx/1.8.1

Ya, I'm the idiot that went to Chipotle in the middle of a launch. Haha

No excuse! ;)

(I love this spirit of this project. If you have any way others can contribute, I would be interested!)

Same here. And https://www.fraudguard.io/ uses the wrong SSL certificate (that of analytics.crynix.com)

Probably the site was taken down by removing the vhost config. So the domain is getting the default vhost and its certificate.

Thanks for the heads up. It's fixed

I have a couple of questions:

- Why can there be only one risk type per IP? What if an IP is a honeypot and botnet. It would make sense to me to have a list of threats or a different value for each.

- Why is the threat level a string? Is it meant to be compared for equality only?

Also your docs need a lot of work. I would like to know specifically what threat types there are currently, what their slug is and what specifically they mean.

Thanks for the feedback, completely agreed. We are redoing the entire docs page to give more info on all responses plus some other generic stuff/updates.

Regarding only one risk type per IP. We set the severity to the max level logged in our system. If it's a 3, 4 and 5 based of attack type, frequency of attack, method of collection, etc it'll be the highest severity logged. We might look at integrating this differently in v2.

Tried putting in quite a few IP addresses from /var/log/apache/access.log that have been trying to hit up "wp-config.php" and "phpmyadmin" and whatnot. All of them came back as risk_level 1.

There are a lot of IPs out there. Unlike Pokemon we can't catch them all. It varies but we run less than 50 honeypot nodes in 15 different countries (because I pay for them out of the kindness of my heart each month as we are not yet profitable) right now that would collect this kind of data. Our goal is if we get more people to signup we will add more nodes. Obviously more nodes = more data.

Why not price a little higher, lower it when you get profits?

We never considered it I guess. Like someone else already mentioned there are other options out there but their prices are insanely ridiculous. Starting at $10 /month the three of us devs/creators feel like a competitive price will keep big and small customers happy hopefully long-term.

Look up articles on pricing plans. $10 is really low and makes your product seem less valuable (if it's so good why are you almost giving it away?). Cheap and free customers are often not worth the headache. You probably want the entry-level plan to be at least $39, $49 or so. Maybe more. Offer a free trial, and let that be enough for the cheap customers.

I don't think there are 100s, let alone 1000s, of low-maint customers that are thinking "hmm, this abuse issue is really a problem on my site, consuming at least an hour a month of my time, but I can't afford $49 to fix it".

Think about it, you're asking for $25 for a million checks. Typically that'll be sign-ups or some sort of interaction. So their volume is probably what, 10-50x times that. Even if they used your API for checking before anon comments, that means they're getting millions of pageviews/visits per month. If such a site can't afford a, I dunno, $199 plan, maybe they aren't worth dealing with.

If you really think you need a charity-level plan, perhaps include a contact link for "open source and educational projects".

Someone will probably point out some wildly successful freemium model. Suppose that's possible too. But even then you'll want to make a large gap between free and premium. No one wants to deal with $10/month business customers.

You know, I remember some old Tom & Jerry cartoon where an older mouse was explaining capitalism. How a factory that sells great volumes is able to reduce its margins and make even more total profit. Something like that.

So... what is wrong with a "low" price?

Must pricing nowadays be all game-theory where you want to extract the maximum amount without any regard for underlying value or actual costs?

It's almost a meme on HN: "you are asking too little!" "Raise your prices, double your consulting rate!" "Businesses don't even notice bills under $4999!"

I'm from Romania where my "business" cell phone (with 1GB internet and basically unlimited calls) is costing me $7/month. My build server on AWS used to cost me $25/month. Nowadays I use my own machines so I only pay for some leftover storage and I get a whooping $1.50/month bill on my card. I pay $39/month for accounting.

No matter how great a startup believes their thing is, a business has to cover a lot of expenses and 100 super-duper-products to purchase do add up. At some point it might even make sense to say: yes, I'll have an employee waste 1 hour each month on this problem instead of adding another vendor/product/contract to the list.

1 million checks a month. If the problem rate is even 1%, that's 10,000 "problems" that need to be resolved. At 1/minute, that's a full-time person on the job! If that's not worth $$$$, the business isn't in the target audience.

And for people using this for fraud, it's gonna take more than a minute, and there might be even more damage. For instance, avoiding chargebacks on 0.1% would more than pay for itself. And that's a good selling point: "Our product will save you $x% a month". It makes it a no-brainer, instant ROI.

So getting, say, $995 vs $25 means he has to find 40x less customers! He can afford to spend a bit on sales. It's a meme on HN because it's true and us engineers have a terrible habit of repeatedly undervaluing things.

He could even offer a "pre-launch" plan if he's worried about startups not wanting to rack up bills before actually having customers. That way they can maintain price plan integrity.

Overall I feel HN/engineers (myself included; I have to force myself here) worry too much about edge cases and keep thinking somehow these cases will make a serious business.

Thats really great advice, we're talking about it on Slack now.

Thanks

As a converse to this....

If you raise your price, you will only end up dealing with businesses that have identified a need for your product, and know exactly how much fraud and other issues cost them.

With your price as is anyone can use it for any purpose, including those that don't involve losing money because of a fraud transaction.

I considered using you for my site just to get GEO-IP. Sure, I could setup a geoip database myself and keep it in sync manually, but at your price point... it was a no brainer.

Plus I'd get threat analysis thrown in so I can know if one of my users isn't able to see the site because Cloudflare is blocking them for being a tor exit node.

But of course if you charge $1000 a month for that, I won't use it for 'any purporse'. I'll just use it for pre-screening new subscriber accounts (and not free ones) because that's the only time it'd make sense.

Look at the numbers again. 1,000,000 checks a month for $25? Even for an ad-supported site (using the checks for commenting) will get vastly more revenue and be able to afford it.

Remember, at $10 for $100, he has to convert 10x the number of accounts. People that are considering setting up their own DB usually don't fall into the category of "good SaaS customers".

True, but if i weren't a technically enabled client then I would want real fraud scrubbing, not this simplistic stuff based solely on someones IP address.

This is something you can add to your own heuristic, but not really use as-is.

Thanks for all the great feedback everybody. We are still looking at pricing. Obviously with all the signups today we will honor original pricing forever if we do decide to make any changes in the future.

Money from nothing project. Congrats!

bad cert?

Can you send a screenshot to hello@fraudguard.io and I'll take a look now?

Reply on news.ycombinator.com