Snort – Network Intrusion Detection and Prevention System | Hacker News Reader

Snort – Network Intrusion Detection and Prevention System

snort.org

117 pointspmoriarty70 comments

To any NIDS newbies out there... Please don't deploy this in-line. Ever. You should always have your IDS out-of-band. Software like Snort is great for detecting threats, but if you block every "threat" you're going to have a bad time. Not to disparage the quality of their rulesets, they are very high quality, but there absolutely will be false positives. I've spent many evenings and weekends troubleshooting IPS problems, 0/10 cannot recommend.

The best option is to mirror all traffic from switches directly to capture boxes where the detection and logging happens. This should be sent to a central system that has a full picture of the network and traffic patterns. That central system should be the one making the decisions, and it should be very smart. Automatic firewall rules should be close to the source, and shutdown switchports should be close to the client.

For safe IPS operation there needs to be several layers of filters, not just a list of "allowed rule IDs". This is the sort of project that takes at least a year to fully roll out - it's not the kind of thing a "security whiz" can set up in an afternoon.

At best, it can be a very useful diagnostic, logging, and threat detection tool. At worst, it can cause very difficult to predict and troubleshoot network problems.

great comment. thing i’d add for folks who want to play with this, follow this setup exactly and turn every rule on but don’t perform any blocking. then review the logs and dial back unnecessary rules. very “fun” exercise in a lab for a student xD

or just use commercial NGFW which uses combination of ASIC and specialized compute cores to decrypt and inspect traffic inline and block threats. They are very capable, but very expensive

Can confirm. I just spent many hours trying to diagnose why an API was timing out. Turns out Snort blocked the IP address of the certificate authority which chrome was secretly using to verify the API's certificate. My local machine had the verification cached but the docker container I was using did not. No indication of what was going on in the top-level network tools; I had to use the hard-to-find network diagnostics recorder in Chrome to even realize chrome was connecting to the certificate authority.

This reminds me of when I used to run http://snortinline.com/ in 2004 - when having the IDS inline seemed quite cool.

This is why we have a SOC where I work. A group that processes alerts and dispatch actions if necessary.

Snort is the hardest piece of software I've ever had to set up. Took me about a week to get a toy system up and running and detecting the suspect traffic I was sending it (I was a dev with maybe 5 years of professional experience at the time, and was considered a linux and security expert).

My friend once got "install snort" as a take-home test for a customer support role (I'm guessing the interviewer never tried). We laughed so hard that it ended up with us recording a parody song about it.

My favorite lines, translated to English, go:

If someone disses me on mIRC

I immediately counter-attaRC

I go and hack into his host

I see his host is "localhost"

He tries to counterhack my box

I catch him with Snort

Yes, yes, Snort is good... I install it on my machine and on _his_ comp too!

> If someone disses me on mIRC

> I immediately counter-attaRC

This is a good stanza! For the non-IRC people, mIRC is often pronounced "merk", rhymes with "jerk".

I mean, it's a 2:30am translation of a Hebrew original. In Hebrew we pronounce it meerk, and the word for counterattack ends in -zeer, so the line ends -zeerk which rhymes well with meerk.

It's not a good stanza in English. I think you misunderstood somehow.

How relevant is a rule based IDS in today's environment?

With most everything fully encrypted, what's left for the rules to detect? If I remember correctly, one of the first performance optimization recommended by snort/suricata is to detect and skip encrypted traffic, to not waste cpu cycles on random bits.

If a malware wants to exfiltrate data or receive commands from a remote command and control, won't they simply masquerade their traffic as regular outgoing https requests and bypass the IDS easily?

Ban outgoing encrypted traffic. Terminate TLS at the load balancer.

For a home network that might be feasible but for a business/shared network you have to be careful about terminating all incoming traffic since you might deal with HIPPA or finance privacy issues.

In my professional opinion requiring insecure internal connections is also a bad idea.

(I previously worked for Sourcefire / Cisco, which owned snort)

You can force everyone to go over a proxy where you can MitM it, or have MitM that swaps out all TLS negotiations to use your own certs and replace things (but then all clients need to have your local CA cert installed on all machines egressing the network.

IPS's have had this downfall for a long while, and likely just keeps getting worse. I think MitM is the only way to do deep packet inspection. Otherwise you're just going to do host/traffic analysis to look for oddities in the hosts being connected to and how much is flowing. Host analysis can give you some data, but no where near as good as what Snort can do with deep packet inspection.

mitm is the only way to secure egress with tls 1.3 native forward secrecy. root ca trusted cert on each device.

ingress is tls offloading at waf

If your WAF is terminating TLS, can't it also initiate outgoing TLS connections? In this case the ephemeral Diffie-Hellman secrets would be created on the WAF, so it would have full ability to inspect traffic.

it won't work for devices/website that do certificate pinning, device won't accept MitM'd certificates, only original non-intercepted certs are accepted

https://www.thesslstore.com/blog/an-introduction-to-pinning/

Right, you pin the cert deployed on your WAF and you should still be able to terminate TLS at it.

I see no issues here - your WAF itself is initiating and terminating outgoing and incoming TLS connections. DHE keys are generated on the WAF. The WAF's cert could be the authoritative one.

The WAF could then re-establish a new connection to applications behind it.

There does not seem to be any technical boundary here.

there are embedded/IoT devices that pin ssl themselves and do not allow you to change that

Just don't use devices you can't configure?

"You can force everyone to go over a proxy where you can MitM it, or have MitM that swaps out all TLS negotiations to use your own certs and replace things"

You've only unwrapped one layer of the onion this way, which only works if the data inside it is in plaintext. If it's encrypted you're back to square one.

it can be blocked as well, because most other benign and good apps dont do double layer encryption

Reasonable plan, until your service needs to interact with an external service (esp 3rd party). Then what?

Perfect security is commonly unusable and therefore often useless, because the business requirements still come in and need a solution.

Is Snort useful at all on a home network level? E.g. for detecting if some insecure embedded device on the network has been hacked and is spamming all the other devices with spray-and-prays?

If not, is there a lighter IDS that would be? Curious to know what the SOTA options are for non-enterprise network security.

The thing with Snort (and others like it like Suricata) is that they produce a lot of false positives. You end up with a process of constant tuning of the filters and the filters update/change over time so that process never ends.

Maybe its not as noisy in a smaller/home environment but you'll still be going through the alerts.

Then when you do get an alert that looks interesting it might take you a lot of time to understand what it means and if it is real or matches the pattern for some other reason.

While you may learn things, and you can block things Snort finds to prevent issues before they happen, you'll have some work to do. Some are up for that work and some are not.

I've had IDS systems show me things I did not know about. I've never had one save my bacon. IDS is one of those 'part of this complete breakfast' things that doesn't stand alone well but needs to be part of a security solution/approach.

It's not as useful this decade due to everything being tunneled over https anyway and it requires you to actually analyse the results. So... I'd say it's not useful for home at all, unless your goal is too learn.

It can still surprise you. A lot of CDNs transfer media over plain HTTP.

Aren't CDN's usually caching legit data though?

CDNs caches will only be as legit as the customers using them. And since moderating content at scale is a hard problem, one cannot assume all CDN data is legit.

For what it's worth, opnsense has suratica as a service, I've spent a good amount of time configuring it and setting up alerting on drops. The only thing it's ever alerted me to is a few malicious domains (malvertising) and my scanner making calls to Google chat (surprisingly cloud print looks a lot like gchat?).

But it happily updates it's ruleset daily, and consumes some 10-20% cpu power from my Intel c2d 3.2ghz cpu. One of these days it might pay off?

More specifically, OPNSense gives you the ET Pro ruleset for free, from Cisco Talos, which normally runs $800+/yr. It does require opting into reporting firewall logging data though.

NB: ET Pro is from Proofpoint.

Doh. You are right. Mixed the companies up.

The first major piece of software I ever wrote and pushed to production was a rules manager/updated for Suricata (essentially open source Snort). As someone who didn’t have a CS background and was self taught, it felt momentous.

I have since left that position so I can’t see the code, but I am sure it was appalling. Even with that, I will also have a warm and fuzzy spot for Snort/Suricata.

I don't know that I would describe Suricata as "essentially open source Snort" since Snort itself is licensed under the GPLv2.

GP was probably referring to the closed rulesets: https://www.snort.org/products

(Former sourcefire / snort employee here)

Both Suricata and Snort are GPLv2

(my personal speculation) Suricata was funded by DHS because Snort was used heavily within the US government and I think they were worried about having such a central tool being owned by a corporation . There were also some issues with how Snort did scaling to multiple CPUs (Snort 2 was single CPU bound, so to run on multiple CPUs, you'd have to start up an instance of snort per CPU). Snort 3 finally is out years later and has fixed this problem.

I'm also guessing that it partially was started when Sourcefire was about to be bought by Checkpoint systems back in 2006 (https://www.washingtonpost.com/archive/politics/2006/03/24/p...), which was SUPER close to happening but got killed for "reasons".

To peer comment: note that many personal installs of Suricata out there still use the VRT rule sets released by the Snort/Cisco teams. Rule sets are where Sorucefire made some of its money (though most was the hardware boxes they sell to corporations). Suricata never got in the business of publishing rulesets as far as I understand.

I used to run snort but I don't like the "buying rules" thing they do.

> Snort is the foremost Open Source Intrusion Prevention System (IPS) in the world. Snort IPS uses a series of rules that help define malicious network activity and uses those rules to find packets that match against them and generates alerts for users.

At IBM's Watson lab, I tried rules and was not thrilled.

E.g., there was no way to know what the false alarm rate was or to adjust it or know what change in the false alarm rate an particular adjustment would make.

And the rate of missed detections was also a problem. For that, for the highest possible detection rate, there is the Neyman-Pearson result, and we should at least try to do something similar in practice!

And to write the rules, it appeared needed an expert in the system being monitored.

So I worked up some solutions, responses to these issues, with some math, and published.

But the people using rules are correct! Rules are what the market wants!

This is one of the oldest problems in network security. Rule-based detection systems converge on antivirus's effectiveness (or lack thereof). But anomaly systems have almost universally failed in practice, no matter what the anomaly model is, and people have come up with lots of them. I can rattle off reasons why model-based systems are hard to operationalize.

Yes, monitoring, anomaly detection, stopping computer viruses, intrusion detection, etc. are old problems. And rules have long been the relatively sophisticated solution technique, i.e., more than just thresholds! And false alarm rate and detection rate have not played much of a role in the discussions.

And attacking the problems assuming knowing some relatively fine details about the system, e.g., some Web server, that is the target of the work and the problems want to detect and correct has been common. E.g., when have an instance of some bad stuff that didn't detect soon enough but want to, analyze that problem or class of similar problems and build a detector just for that. And, yup, for that particular problem, might have a nicely low false alarm rate and relatively high detection rate -- on this class of problems seen before, analyzed, and understood. So, here maybe should say that have built a model.

But I was not impressed with rules, didn't want to need a lot of fine expertise in the target system, and wanted to detect also problems never seen before, with some good progress on rates of false alarms and miss detections.

Yes, what I did is not easily called "model-based" and was supposed to be "new, correct, and significant". Maybe it was at least partly new; maybe it still is.

I started with the data being random variables -- that's not assuming much! Then would like to have the assumptions of independent and identically distributed (i.i.d.) random variables, but for practice that is asking a bit too much! And for some positive integer n, having i.i.d. random n-tuples seemed a bit much!

Well, I used something weaker, but still strong enough, with some math, to get some results, called exchangability.

So, right, my techniques are distribution free, that is, assume nothing about probability distributions and make no attempt to estimate such. Gee, I certainly didn't assume a Gaussian distribution!

I was able to get false alarm rate adjustable and known exactly in advance. Looked like progress to me!

I had some real data, fed it into my software, and, presto, bingo, the false alarm rate was as adjusted and desired!

And using a classic result of S. Ulam, I was able to show that for detection rate the math was not trivial, that is, was better than merely trivial. More progress?

To use the work, don't have to be an expert in the system being monitored, don't have anything like a model, and get to detect problems never seen before, sometimes called zero day problems.

So, just have some random variables, exchangability, and the Ulam result.

But, you are correct: Rules, models, thresholds, etc. are what is popular; the Ulam result is not very popular!

My paper has been in the literature for many years now, and I suspect that so far the number of readers is 0.00. Not really popular! Not likely to compete with the new Tom Cruise movie!

Ah, I forgot: I also worked up a relatively fast algorithm for the core calculations!

I learned my lesson and am doing things quite different now! I started with a problem some hundreds of millions of people would like solved!

At Easter, I talked to a guy who works in computer and network security. I asked him what would be the security risks if a computer with an up to date instance of Windows Server as a Web server just openly faced the Internet? His reaction was that the results would be multiple disasters from attackers around the world. He mentioned "SQL injection" -- gads, I thought I was quite careful when I took the data from the user and constructed my SQL queries!

So, near the top of my TODO list is just to write a little .NET code, listen on some IP ports, scribble what comes in to a file, and see what I get!

Anyone want to predict what I'm likely to get?

You know if I was a state sponsored intelligence unit the very first software I would backdoor would be IDS and TOR and every type of pentest tool. These tools secure you from intrusion like a flare gun hides your location.

Not everyone has the same threat model and goals.

Snort was the 2nd thing I setup after big brother when I was a sysadmin in the 1990's at an ISP.

Big Brother was the shit! It was amazing for a small IT shop to have a central view of everything in the environment. We setup basic up/down, disk, cpu, memory only and then every time there was a surprise incident asked “how can we monitor for this?”. In a fairly short time there were very few surprises…

Does anyone know of a good snort alternative? Any recommendations for a company that runs mainly off AWS?

AWS network firewall loaded with suricata rules?

I've been exploring this exact possibility a few days ago. The support for Suricata's rules in AWS Firewall is pretty limited though (https://docs.aws.amazon.com/network-firewall/latest/develope...).

AWS is a totally different beast and you'll have other useful tools in the stack there. Cloud trail, guard duty, VPCs, proper least privilege and IAM rules, etc. What are you looking for in a snort alternative? It is the open source standard; though you might also look at suricata. Perhaps a whole security stack like security onion which incorporates snort and a lot of other tools. Might be overkill for what you want though.

> Cloud trail, guard duty, VPCs, proper least privilege and IAM rules

We use all these. I guess this could be considered IDS + IPS?

I wouldn't put that level of logging as the same as an IDS/IPS, just alternatives.

Not IPS, those tools don't block access.

Not sure about the AWS bit but I like Suricata.

I've set up Suricata (similar to snort in many ways) on AWS, and it was a pretty horrible process.

Basically what you need to do is set up VPC-Mirroring on every single interface in that VPC and send all the traffic to an endpoint attached to your snort server.

https://docs.aws.amazon.com/vpc/latest/mirroring/what-is-tra...

It's a ton of work, and is quite expensive, since you're paying per interface + per traffic.

A better way to handle it (imo) is to just enable VPC flow logging and pull the cloudwatch stream into your SIEM. Suricata/snort isn't that useful for threat detection anymore (thanks to HTTPS), so the important data to capture is just firewall logs.

You can still have suricata/snort, but you have to be more selective with it.

> A better way to handle it (imo) is to just enable VPC flow logging and pull the cloudwatch stream into your SIEM

Thank you. Any recommendations for SEIM for a small company?

I'd say just spin up a SecurityOnion stack. It's essentially a "SOC-in-a-box". I had a proof of concept machine spun up and generating alerts off of replayed PCAPs in a day.

check out managed instances like azure sentinel

check this https://aws.amazon.com/marketplace/pp/prodview-mn63yjbq37n4c it is expensive, but for large company with good infosec budget it is worth every penny

My first job after college was working on Snort 3. Glad to see it is finally released.

Did you know that Luca Deri NTOP project was intended first as a NIDS?

I don't know that it started out that way, but it definitely was (and still is) a network security tool for people, and was part of a mid-aughts zeitgeist of flow-based detection tools.

Please change title to "Snort 3.0 - Network Intrusion Detection and Prevention System" to make it clear there has been a new release.

Reply on news.ycombinator.com