Elastic SIEM – Security Information and Event Management

Heard the same about elastalert that it is difficult to manage. Any idea how much time/effort is spent per day/week/month to manage the elastalert rules and what level of expertise is required?

A SIEM is one of those things where you'll know if you need it. Most of the companies I know using a SIEM do it because audit and compliance requires them to have one.

Just heard about this today at reinforce conf. Looks pretty interesting and integrates well with cloud logs or on prem. Definitely going to look into it more.

I grew up on Splunk and can’t seem to figure out how to get the same level of aggregations and analysis of ad hoc data out of ELK. Sometimes I think I’d be better served with Jupyter and Spark or similar.

SIEM basically aggregates and collates all security related events, logs, and alerts across all your network gear/servers, etc.

Security Event Information Management system

In my day job I'm a security consultant and I work mostly with SIEM technologies. The other comments are right, but maybe I can give a bigger view. Basically you send syslog from your Linux servers, Security Event logs from your Windows server, access logs from your critical infrastructure, authentication logs from Active Directory, IDS/IPS and firewall logs from your firewalls, etc. The SIEM collects all this data, but what separates a SIEM from a simple log aggregator is the intelligence it uses. Many popular SIEMs have rules you can define (or are pre-defined) that fire alerts when a potential security breach is detected.

For example, if someone hacks your Internet-facing web server, your IDS might detect that. They then brute force the password to your production database server, which Active Directory might see. They then use nmap to trace your internal network, which would show up on your internal firewalls. Then they hop server-to-server until they get to a critical server. They then download a payload, infect that server (which your AV might pick up) and start exfiltrating data (which the firewalls and proxy might pick up).

You have all of these security tools, but without some intelligence linking all the logs together and correlating the data, you're stuck tailing these logs individually, hoping you catch the right log at the right time and can remember everything you've ever seen. And we're talking tens of thousands of events every second. A SIEM takes all of that data, does the searching for you, correlates all of the events across different security technologies and vendors, and alerts you when it detects someone doing something they should not be doing.

Popular SIEMs are tools like Splunk, QRadar, ArcSight, LogRhythm, etc.

It's expanded in the first screenshot. That's it.

Ok, we've acroexpanded it in the title above to take pity on everyone else.

It's more than just log management. SIEMs usually implies a case management system and some kind of rules based system for correlating different events into a case/incident, thus the name.

Also, just as an FYI, it's pronounced SIM.

It is two technologies combined: SIM (security information management) & SEM (security event management) == SIEM

The SIM came from the requirement to collect events (operations).

SEM stems from the desire to monitor / detect (security).

When we (the operators & security people) realized we where both collecting sort of the same data, the plan was hedged to combine resources and build something that satisfied both our needs. Marketing realized this as a great way to pitch a story that actually made sense to everyone involved: the SIEM market exploded.

Search has been suffering under Elastic for a long time. Only a small percentage of Elasticsearch users use it for search. Then again, Elastic employs many Lucene committers, so they indirectly help search by being a major maintainer of Lucene.

If you want search, most of the NLP crowd is using Solr.