Comment by pilooch | Hacker News Reader

Comment by pilooch | Hacker News Reader

Machine learning is pervasive in the new products targeted at cybersecurity and infosec. From malware to anomaly detection among billions of logs, it's been part of the demand of my customers for at least four years now. Most of the largest companies are already running and testing a series of novel proprietary or custom in-house systems.

The main differences with straight 'supervised' machine learning are the lack of labels, and the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here). So unsupervised learning is king in this domain, and often security operators have to keep an eye on and interpret the results. Another difference with other fields is that datasets are rare, mostly because of privacy as the logs can be very revealing. For this reason the market exhibits a lock that cannot be overcome by everyone I believe. Basically, you sort of need to be in the place already.

Here is a recent very high level survey on the topic (a TC report, not that bad for once), https://techcrunch.com/2016/07/01/exploiting-machine-learnin...

Here are two of our own Open Source tooling we use the most in application:

- very efficient C++ map-reduce feature generation for logs (for ML and analytics): https://github.com/soprasteria/cybersecurity-miw

- machine learning / deep learning server: https://github.com/beniz/deepdetect

The ML cybersecurity + infosec field is still young, but moving very fast, a lot of new startups and (somewhat opaque) products.

Great summary! By nature of my job (eng lead of a major mobile malware detection team) I have a lot of startups pitch their ML solutions to me. A couple of thoughts:

- There are no publicly available data sets for training available. There are a few small ones and a few old ones, but they don't reflect the reality of 2016. Companies that approach me and pitch me solutions to the malware of 2012 are not useful.

- The majority of mobile malware is based on some kind of social engineering. On a code level these are indistinguishable from legitimate applications (the same APIs are used in the same fashion). The only difference is whether app behavior meets user expectations or not. Making this decision automatically seems intractable so far.

- Malware is not really a well-defined term. There is phishing, toll fraud, Trojans, privilege escalation exploits, ... If you generically look for malware, the signals you will look for are going to approach the complete set of APIs made available by your OS. Your results will just be a giant blob where everything is connected. Pick a single malware category and focus on just that at a time. ML signals for priv esc will look very different from those for phishing.

- ML is sexy. Malware analysis is not. Startups seem to hire too many ML people and not enough malware analysis people. I've had startups pitch to me that had literally zero people on staff who knew what mobile malware actually looked like. They just did anomaly detection and then tossed the results over to my team to verify the results. That's not how it works. We're not your QA team. :)

Hey, just finished a malware ML custom system for one of the largest european corporations, large enough that some malware is targeted at them. Result is 97% accuracy (they did retrain and check on their own held out dataset). More careful analysis is needed (many malware have high entropy 'zones' that may help the classifier find the right category), but overall it does work.

See the Microsoft / Kaggle challenge on classifying malware families, winning solution is > 99% accuracy IIRC.

The 99% means little when it suffers from a similar sort of problem that the immune system has with cancer. Adversary's lack of stationarity vs a fixed model.

That's what the research under the banner security via diversity and "moving target" are doing. I recall the Hydra firewall from Sentinel did that sort of thing. OpenBSD and grsecurity do in OSS for parts of their OS. Such methods can be combined with these.

Interesting name. Reminds me of a security scheme, Symbiotes, I briefly evaluated on Schneier's blog. Injected security into legacy, embedded applications with various tradeoffs. Where did you get the name from?

Can you describe a security setting where 97% accuracy is actually useful? Unless the events you're looking at are low volume or you somehow have much more malicious data than everyone else that seems like a recipe for your results being primarily FPs.

For context, a company can easily get ~1B security-related events a day, so even reporting say 0.1% of those wrong a day means some poor junior analyst has 1,000,000 tickets to slog through. If you expand that to full packet captures as suggested in the article... ouch.

(We do some cool visual analytics work here, including unsupervised learning / classification, and target more of the problem of "given an incident you're already investigating, what else should you now look at from across all your tools?")

We're talking hundreds of thousands of malwares here.

Focusing on your last point: this is true in a lot of fields.

ML is amazingly powerful, but if you don't have sufficient domain knowledge, or you aren't collaborating very closely with actual experts, you can make very dangerous mistakes. Domain knowledge helps a lot - not just in malware, but in biology, image analysis, etc..

Yeah my initial reaction was...wait a second ML is used in info sec. The most obvious examples are intrusion detection and general network analysis. A less glamorous example is pretty much every spam filter (which also filters fishing stuff and is thus very much infosec-relevant).

I think false positives are a huge problem in the info sec context and less so in other domains. To give a made up example let's say you have ML-based intrusion detection that is hotwired to some sysadmin pager. Even if your false positive rate is fairly low (at acceptable levels in other domains) with the sheer amount of data a bunch of sysadmin calls will be triggered in sum. That is a huge human factor risk as it most likely will result in teaching admins to ignore certain alerts....which of course opens up the interesting attack vector of understanding/"reversing" the ML and triggering false positives a bunch of times until you want to attack through a channel that will trigger a similar response.

Regarding the available data sets: I suppose honeypots/-nets could be created for the purpose of gathering some data sets.

Is there any evidence that this works in the sense of foiling sophisticated attacks?

AFAIK not really. Detecting C&C is not that easy in unsupervised settings. The MIT paper (link in the TC article) makes claims, but this is mostly PR.

There are some reports from Google as well that you may find around, and at least at the time (few years back) they make the same observation we do with my coworkers: the rate of false positive remains high, and thus it is the job of trained security operators to spot the sophisticated attacks.

Empowering the operators with more data mining tools is a good step not yet fulfilled I believe.

If you ask "Is there any evidence for ..." for so-called IT security products then the answer is practically always "No".

Unfortunately the FP rate is just unacceptably high, you end up overloading your SOC (which means your analysts end up flagging a lot of real threats as FP)

And of course Cylance for file artifact analysis and TrustPipe for two way network conversation analysis.

> the unreal volume of data (cheaply produced by machines, in machine time, we're talking microseconds to milliseconds here).

Unrelated, but that gave me a thought to ponder: Will it ever be possible to use computers made inside this universe to model a MORE complex universe?

Has that already happened?

Well, what do you mean by more complex?

A Turing machine can simulate a python interpreter.

Have you seen the xkcd with the rocks in the desert? https://xkcd.com/505/

I meant create a new, virtual universe that is more complex (i.e. contains more information) than this one.

Well, in order to store the state of the simulation, it would need to store the amount of information needed to describe the state of the simulation. But the information needed to describe the state of the machine doing the simulation, by being enough to describe the state of the machine storing the information about the simulation (including the part of the state which stores the information describing the simulation) , is therefore enough to store the information describing the state of the simulation.

So, therefore, the smallest possible complete description of the state of the simulation is not larger than a complete description of the machine doing the simulating.

So if that is what you mean by more complex / what you mean by more information, then, no, the less-information thing cannot completely simulate the more-information thing.

It could be that our universe's state can be described by the initial input '1234567', 1 page of differential equations, and an amount of time T to simulate.

An inner universe's state could have an initial input of some string describing all of human history up to T in the outer universe, the same differential equations, and the same amount of time T to simulate.

One might say that the inner universe contains more information, because the smallest unambiguous description of it is larger than the smallest description of its parent universe. However, the outer universe is able to simulate the inner universe.

But you can specify the information of state of the simulation by specifying 1234567, the differential equations, T, and something to identify the machine that you mean, and that you want the information it stores. This gives you the same info, and would presumably be shorter than "some string describing all of human history up to T" . It would still describe all the info though.

The purpose of 'Some string describing all of human history' is to have something irreducibly dependent on the first machine. It's extremely unlikely that you'd be able to find an alternate way to describe it that didn't involve describing the outer machine in some way.

You could mark a time and an algorithm for accumulating information, but you now have 2 more pieces of information in the inner machine than the outer one. So the outer machine has somehow managed to simulate the inner machine even though it has less information.

Although... the moment you actually carry out this experiment, it falls apart: Instead of using T, you can just look at time T + U, where U is the amount of time needed to simulate the inner machine. This is because we're sitting here talking about the experiment and eventually carrying it out, so those initial conditions encode the conclusion of the experiment.

This is subtle enough I'd have to look at the math. I can't think through it.

Do we know if it's possible (or impossible) to run a perfectly accurate, 1:1 simulation of our universe inside a computer?

If yes, then can we add something to that simulation, like an extra type of particle?

We already do quantum simulations at tiny levels and at a fraction the pace of real time—so, yes, assuming we can quantify the rules of the simulated system.

Reply on news.ycombinator.com