How to evaluate a cybersecurity vendor's ML claims

This is a little silly. ML is very good at building things much more general than rules.

As a person that’s built and deployed a lot of ML solutions in a related space and brought a product to market meant to help others do the same, we consistently ran into two main camps: 1) people that had a long history of rule writing and were unreasonably closed off to the idea that ML could help and 2) people that thought ML was magic and could do things it most certainly cannot.

I understand the sentiment of being skeptical in a world of outrageous claims from “fake it till you make it” startups, but the idea that most ML is not better than rule based systems is just wrong. You need both and they solve slightly different problems.

I tried to make a ML system to detect malware on a network. Kept coming back to what is basically the second and fourth points:

Rules and honeypots work better and more simply. A machine learning system will need to learn from something, continuously because malware will adapt to what the model learned previously. Every time you figure out a way to make it classify some traffic as malicious and other traffic not, you're basically writing the rule, except it's like beating a particularly retarded rat brain into figuring out and writing the rule for you.

For reference, the 2nd and 4th points are

> How do they know their test data is anything like real life data? How do they monitor system performance in the field?

> Ask them why they didn't take a simpler approach than the approach they took.

Making malware trigger rules is simpler and basically the result of #2.

Have been on the other side of that call and those questions. I had straight answers for each of them.

But if I worked in a bank, I wouldn't use an ML behavioral security tool to stop bad guys, I'd use it to identify, model, and detect top traders so I could double down on their bets. Seriously, if the model actually worked, why would you waste it catching hackers? Arguably, if the underlying tech is not useful for that, it's not useful for anything.

What it comes down to is, "the confusion matrix is the product." The mistake ML tech companies seem to make is thinking that their product applies to a given vertical (banking, health, govt, etc) , but in fact the balance of that confusion matrix applies to companies whose growth stage defines their needs, imo.

This headline reminds me of "Time AI"

https://www.vice.com/en/article/8xw9kp/black-hat-talk-about-...

https://www.theregister.com/2019/08/26/black_hat_sued/

Working in consulting with a lot of these products, I don’t really see ML claims as indicative of anything one way or another. I know some pretty good products claim to have advanced ML and some better ones that don’t talk about ML at all, and some really bad ones on all parts of the spectrum. Rather than focusing on ways to get a gotcha on some poor sales team that probably doesn’t control the language used in their marketing, if you want to evaluate whether an EDR product is able to detect things the best way is probably to skip to steps 7 and 8 and ask them about testing - has the product undergone a MITRE evaluation? What’s their strategy for discovering new tactics and validating that they are detected? If you’re evaluating the product on a live demo, your red team should ideally be giving a decent battery of tests, not just firing a macro-laden Word doc and Metasploit...even traditional antivirus ought to be catching that stuff in $currentYear, though of course many don’t.

ML and security is tricky. Its a good tool, but shouldn't be the only one. With threats constantly changing, ML alone will fall short. Its great for so called 'easy' stuff or locating 'odd' stuff to prioritize.

On top of that you need explainability and reproducibility. Depending on the ML in use, this can be difficult. When a FP or FN occurs, customers want to know what happened and can you fix it. They don't like to be told things like "because the ML said so." Reproducibility is important for support. When things go wrong, they need to know how to right it. If you can't explain or reproduce, its hard to say the misclassification is resolved and won't happen again.

>3) Ask them where on Wikipedia you can read more about the approach they took. If you can't read about it on Wikipedia, ask them where their paper is in the peer-review and on arXiv. If the paper doesn't exist / is a "trade secret", deduct 3 points

Is it common for ML companies to publish their algorithms?

This seems like a good way to evaluate a Sales Engineer. With a couple of decades in this industry I've noticed that the sales engineers are just as likely to give an answer about their previous company as the current one.

If they use discriminators like K-Splice, Munkres, Bloom, Kleene Hierarchy, Markov-Chain to provide refined feedback into their ML, then there is little reason to doubt that ML is being used.