Radar – A new set of integrated tools to help prevent fraud
stripe.com
stripe.com
Thanks to Stripe for making it not-a-black-box! I hope others who build machine learning systems also find a way to make its decisions understandable by humans (when possible).
I agree however that to my eye it does usually seem excessively black-box, as it's not like most fraudsters are idiots, they already know what tools are arrayed against them.
Rather, it's that these predictive models mostly consist of linear combinations of metrics. (By necessity: "linear combinations of metrics" is usually what most accurately reflects the real world.)
If you know exactly which factors, at what weightings, a given predictive model has, you can build a simple function to spread out your fraudulent activity that will ensure that you never exceed their fraud threshold in your aggregate score.
Say someone's anti-fraud model uses two factors: P(using a prepaid card) and P(using a Nigerian postal address). If you know the weights on those two factors are (0.3, 0.7), and you know that the system says "fraud" when you score above 1.0, then you can still do both of those things, just so long as you never do them both at the same time.
Basically the only thing that makes these systems useful at all is that their input-factors and weights are hidden information. There is no anti-fraud system that would survive being public; they'd all, as a class, be rendered instantly useless.
MaxMind isn't a black box either. You can pay 0.03 USD to get the full break down of scores on an inquiry. You can register and get an ID number to token your fraudulent VM/RDP with the card before submitting a real payment. That partially helps defeat their device tracking [1].
That leaves a trail.
You want to create a history, a trail, so to speak with browser/device, IP, and card usage. MaxMind's device ID tracking is similar to what Radar is offering. You've used your card before on a MaxMind protected website with device ID creating LSOs. Now they have you fingerprinted and they can tell if you suddenly start making purchases from an unusual PC/IP. It helps you as a legitimate cardholder at the expense of your privacy.
Stripe uses IESnare when you sign up to determine if you have any sketchy internet history or are a previously banned user. It's a similar strategy, pose as a legitimate user for a week by browsing the internet normally and search for payment processors to "compare" online. Then sign up after you've created history on the device and IP.
This has become clear now....but Stripe has thus been blocking legitimate payments from our users with no insight for us
80% of our clients are travelling at the time of purchase... Now we understand why we were getting so many BLOCKED transactions!!
Although this is a welcome addition, a heads up would have been nice...
This is a big step for Stripe. I've often asked why they didn't have an integration with MaxMind or SiftScience already set up. They've been building their own behind-the-scenes the entire time! This feature is fantastic if you are a merchant and want to avoid fraud.
To me, the more interesting side of online credit card fraud is the merchant/payment processor side. Stripe has a cult-like following in the fraud world because it's known as the the easiest target. They make it so easy to sign up and process transactions compared to other services like Authorize.net/BrainTree/etc. They've shed this label recently, in part because the biggest forum thread discussing it was closed. The other reason was because it became so much more difficult. With this release, I think it's simply because they could identify accounts with high numbers of suspected fraudulent transactions. All the fraudsters were used to just signing up, running charges on their webstore with sock5, and waiting 2 days for bank transfers. Now Stripe can identify those transactions well in advance and assign each account a risk score. Previously, Stripe had to identify the account risk by sales volume, chargebacks, bank account provider, sign up IP, and every one's favourite privacy invader IESnare.
Fraudster's have one last shining hope against Stripe. Passing their card data to Stripe via API, instead of Stripe.JS/Checkout. Radar only works with Stripe.JS/Checkout. Setting up your own web server to pass card information prevents them from ever seeing any IP address except the web server. All you have to do to get them to be okay with this is to turn over a PCI self-compliance form. Rumour on the internet has it that there's a pre-built web application specifically for charging Stripe accounts via API.
I'm still looking for a job in fraud prevention friends at Stripe :D
Edit: I lied my email is rwmurray @ [vt.edu].
If you strip away the "buyer's" IP then a lot of their ability to detect fraudulent transactions goes away. They still have account based limits and other methods, but my personal opinion is the anecdotal increase in difficulty of creating fraudulent Stripe accounts is due to Radar based detection.
what was that forum?
Threads deleted, but only to stop newbies and methods from leaking out to public readers like myself.
It seems one of Stripe's biggest risks is the impending PSD2/XS2A changes within the EU/UK. This means banks/merchants/retailers will ditch traditional card networks (and their fees) to instruct P2P payments directly. This probably opens up a host of very effective anti-fraud measures too (eg. 2FA with mobile devices).
I wonder how Stripe will react to this major change in the market?
For example: https://developer.americanexpress.com/products/accept-amex
Probably not, as they are quite US-focused and 3D-Secure is still in closed beta. Probably better margins in the US.
I can't wait to see how they expand as a company going forward and would absolutely love to work with them if I wasn't preoccupied with more personal pursuits.
How would I ever know if the rule I've built is too constraining, or too loose in accepting payments?
Payment is not exactly an area of my business that I want to do a lot of trial and error..
If you do want to write custom rules on top of what the models are doing, we've actually built in a testing interface to the rule creation process. When you test a rule, we'll actually simulate what the rule would have done had it been active for the past 6 months. Using that information, you'd be able to tell the # of legitimate, fraudulent, or already blocked payments that would have matched the rule & make a decision on what's best for your business.
That said, we're looking to make our opinion on a given rule more clear (rules are still in beta) and would love more feedback on how we can make this better. Feel free to drop me a line (tara@stripe.com) if you have feedback!
That is slick.
To do this more precisely, a scoring rule (https://wiki.lesswrong.com/wiki/Scoring_rule) gives a system credit for both (1) making accurate predictions and (2) being confident at the right times.
E.g. if we are executing a charge for a known-good customer, but using acompletely new card - we'd like to suppress all automated fraud checks and, ideally, indicate to the client's bank that this is a legit charge.
We try to be smart about things. For example, if you use Subscriptions, we assume that subsequent charges of a happily paying customer are also very likely to be good and so we do not block them.
Most users won't need to tweak the default behavior, but it is flexible in both directions. You can use the same rule builder interface to whitelist a transaction vis-a-vis Stripe, using boolean logic on a variety of things Stripe knows about the charge. Whether this will work for what you want to do depends a bit on the specifics of it -- we'd be happy to advise.
A subtlety: Due to the way credit card payments work, customers' banks already assume that every business running a charge is representing to them "It is our good faith belief that this payment is legitimate" and so they don't expose a way to say "No, really, we're EXTRA SPECIAL sure about this one. Please give us their money." If they come to the conclusion that a payment is likely to not be authorized, they will take action to protect their customer.
(Disclaimer: I work at Stripe.)
That's insane.
If Stripe is sure that their models work, they should offload the chargebacks from the merchants.
A friend of mine worked for a startup that did exactly that. They were sold to an online payments behemoth in about 2009.
But I'm not wild about the moral hazards of insurance. Merchants should always exert some customer-selection hygiene.
Well in fact their is an issue of misaligned incentives here. Your chargeback insurance has a strong incentive not to receive any chargeback (of course), so they will be overly cautious and decline a lot of valid charges. Stripe on the other hand as perfectly aligned incentives with the merchant, as they don't make money on a blocked transaction.
(disclaimer: I work at Stripe)
(Congrats, we were using a separate fraud detection company that was quite intrusive and this seems much better)
Edit: It appears there's a small per-transaction charge for their enterprise customers on custom plans but it's now included for free with the standard pricing. Can anyone confirm this?
Ironically, we probably have much less interest in 3-D Secure if Radar will now let us deal with the same "sudden spike in dubious sign-ups" problem anyway.
One interesting possibility might be if the kind of rules you allow for Radar could be used to determine whether to apply 3-D Secure if it's available, so we could enable it selectively for countries where people don't already have a zip/postal code to verify for example.
However, for my own small businesses, I suspect we'll be more interested in what Radar can do, particularly to add immediate extra security checks if we are suspicious of a pattern of requests from a certain part of the world.
My theory (just a theory) is:
A. banks fraud detection systems just don't like the way stripe is avoiding the whole merchant account game, so stipe just has a bad rep at banks
B. stripe is so easy to set up and use, therefore various stripe websites has fraudulent transactions going on, so from the bank point of view just being a new merchant coming through stripe is automatically high risk
The whole machine learning for fraud detection imho is very annoying from a merchant point of view, because there is no real way to fix it. We have had cards that got declined only the second-third month in a rent-to-own business and the only thing we can do is to ask the customer to call their bank.
Hopefully once fraud detection happens at the stripe level instead of the bank level, banks will see less fraud coming from stripe and the overall decline rate will ... decline.
We see quite a lot of failures on the second month of a subscription, particularly for customers abroad (we're in the UK). We also tend to see these sort of failed charges happen in waves, with few problems for a while but then a sudden spike in declined transactions. That suggests at least one plausible explanation: the lack of CV2 after that first charge may be enough to tip something over the edge into "too suspicious" territory following some sort of update in the overall scoring scheme used by whoever is blocking the charge.
We're looking into various potential ways to improve the situation, such as allowing more automated retries of failed charges over a longer period before we give up, and possibly advertising prices in our customers' local currencies. However, each of those has some potential downsides and obviously there is only so much you can do as the merchant anyway.
If Radar can start to make Stripe, and by extension its merchants, a harder target for for fraudsters, maybe that will also move the needle a bit. A few times I think we've lost more subscribers in a month to failed card charges than everything else put together, including customers actively choosing to cancel and including failures via all other payment methods we accept, so it's definitely an issue worth exploring.
That's a feature. It's a horrible system which I wish nobody ever used. I've literally never been able to successfully complete a transaction.
Even if you're in a normally low-fraud market, as every business I work with that takes card payments is, almost anyone taking small value card payments on-line is vulnerable to being used as a card testing engine if their system can be automated. If nothing else, being able to toggle the extra check on temporarily if you become aware that your system is being abused that way is an extra level of security you can apply in a hurry.
Personally, I will never use a 3D Secure system these days. If you require it, I'll simply skip purchasing.
But in France and the UK it's pretty standard; cardholders are used to it, and issuers make an effort to decide whether it's worth requiring authentication.
most are buying credit for online services, like online data storage, care.com, etc. and to throw a curve ball, they created yahoo account with my first name plus extra characters and created a subscription to USA today, verified with my fake yahoo account, and sent it to my home.
it happened after I went out with friends to a bar and they had my Amex and ID on a tap.
Cards stolen online typically come with the CVV2 and billing data that makes online fraud easier. At one time the Secret Service would analyze patterns in purchases to identify which merchants had been compromised. They wouod identify large databases of cards that had been compromised and would turn that information over to the card issuing bank. Given the cost of reusing the cards most banks would chose to mark the cards for additional surveillance instead of re-issuing the cards. At a conference one bank executive said that 20% of cards in the portfolio were marked this way.
The net is that if you carry five cards it's probably that one is in the hands of a fraudster already.
Fraud is not an issue at all for our business SaaS products.
For instance, fraudsters that have received a bunch of stolen credit card numbers will want to test which cards are still good. They'll do that by making a small purchase at an online retailer, and if the charge clears, then use the card to get something illegitimately. While you might not be a good target for that second step, anyone that takes credit cards and has weak fraud protection steps will be a target.
What one day looks like an unexpected spike in sales becomes a huge spike in chargebacks later, as all of those sales were fraudulent. I am pretty sure there's been stories in Hacker News about this.
The scammer orders the item, using the billing address of the legitimate owner of the card, but a shipping address that corresponds with a freight forwarding operation. These places offer a US address, but forward the shipments on to various places, including South America, islands in the Caribbean, etc.
Typically, the legit card owner notices the transaction way after it's shipped, and files a chargeback. As with all "card not present" transactions, the shop owner then foots the bill, including a chargeback fee.
It took about 3 times getting burned before we took the time to put some countermeasures in place. The most helpful were geo-ip location for the purchase itself, and a "flag this for review" filter for anything going to the Miami area and/or anything with a longish suite number, keywords (freight, global), etc.
Edit: Worth noting that most credit card fraud solutions I've seen don't have a way to take the shipping address (as opposed to billing address) as part of the data to check. For the type of fraud noted above, it's vital. Geo-ip doesn't work well when the buyer is using a US proxy, or US based accomplice.
The government-owned postal service in New Zealand, NZPost, run a service known as YouShop https://www.nzpost.co.nz/tools/youshop specifically to allow Kiwis to get access to products that are either only sold overseas, or would cost ridiculous amounts of money to ship via those sellers. They operate by providing a personal address in US, the EU, or China, backed by third-party freight forwarders.
Additionally, it creates issues even with legitimate customers. If the product arrives with shipping damage, there's no way to know if it was caused by our shipper, or the freight forwarder. The customer, though, is quite sure it's our issue to resolve...
That said, it can also be defeated.
What sucked is that the charity would then be charged a fee when the card holder reversed the transaction.
This costs the charities a lot of money in chargebacks and man power in dealing with it. Stripe's offerings here could prove invaluable in this scenario.
Thanks to uladzislau - wasn't aware of SiftScience - will have to check them out...
That's a big probably.
"We pinpoint fraud by building behavioral signals from across 100,000+ global companies."
Apparently they do ;)
They're processing around 100M API requests a day. https://twitter.com/patrickc/status/788752160284487680
Because they have their payments dataset, they're relying on having access to more non-public information to better train their models on.
- Bitcoin: https://news.ycombinator.com/item?id=9077293
- Open Source: https://news.ycombinator.com/item?id=10167198
Nothing beats manual verification. People aren't sharing credit card numbers on public forums and mashing them against Stripe. People are paying for fulls, and grabbing a socks5 that's piped within a few miles of the address of the cardholder.
Never trust your processor to protect you against your (potential) customers. Stripe has very little incentive to do so. They'd rather you pay that fat $15 fee when you get hit with a chargeback. They really would.
I'm coming out with a book about Stripe (and a few other processors) and fraud. Trust me it will be good, and this is already a part of it.
Sincerely,
Someone who was once your enemy
PS my favorite part of this? Telling the carder how to defeat their algos:
* "This card has been used from an unusually large number of IP addresses across the Stripe network over the last 24 hours."
* "This email has been linked to an unusually large number of cards across the Stripe network over the last hour."
Thanks for not saying the card was declined. If you wouldn't mind, please hold while I switch socks and make a new email.
Sorry if this is crass, but whoever decided on telling the end-user why a card was declined... complete fucking idiot and should never work in fraud protection or payment processing again.