beamy

For me, so much about building a traditional classifier goes into measuring and improving its performance on the data you’re making decisions about.

With Jev, we seem to have just ignored all that. There seems to be some magical thinking that, because it’s AI, its decisions must be accurate.

Which I don’t think is really justified given the narrowness of the benchmarks and breadth of tasks people want to use it for.

> We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities

But aren’t calibrated predictions one of the defining features?

This is a good explainer: https://magazine.sebastianraschka.com/p/claude-watermarking

Isn’t this like arguing we shouldn’t have food standards because people should be free to decide what they put in their bodies?

Sure, in principle that’s true.

But in practice, is it really fair to expect everyone to understand the health risks of every possible ingredient?

Likewise here, is it fair to expect the average consumer to understand what it means to host a residential proxy? Or even what a residential proxy is?

In my experience, what you're describing would more specifically be called Fraud Prevention rather than Fraud Detection. Both tend to coexist and are complementary in a mature setup.

For Prevention, you're always going to be constrained by latency requirements, available data and an incomplete picture of user behaviour. You make a quick decision using ML and rules that deals with the majority of cases. But those constraints make it impossible to precisely prevent all fraud.

Detection deals with the downstream consequences of this. A team of analysts will typically analyse the accepted transactions for signs of fraud. This is particularly important for fraud types where you don't get an external signal like a chargeback or customer complaint. Platform integrity is one such example. But Fintechs will also see this building anti-money laundering systems - you need to go looking for the fraud. This is the process the article is describing.

I say they're complementary because the detected transactions become the labels for training and evaluating the next iteration of prevention models.