Zeek is now a component of Microsoft Windows | Hacker News Reader

Zeek is now a component of Microsoft Windows

corelight.com

127 pointsrapnie74 comments

I had a chance to use Bro a few years back in a network traffic analysis software and the code was fairly bad - global states, no multithreading, weird scripting language (because of which everything was dynamically allocated with measurable overhead). We ended up implementing our own traffic analyzer, during which we found that major protocols implementations in Bro had bugs or failed to detect or parse valid traffic.

I hope they got better over the years, if they want to integrate into such major products...

Well, putting it on the endpoint will actually make scalability much less of an issue and allow them to get away with much more sloppiness. So I'm not optimistic.

They operate in completely different ways, but depending on what you're looking for, you might want to look at Suricata, especially for protocol decoding (a good chunk, especially the decoders, are built on the AMAZING `nom` parser-combinator crate in Rust and super extensible if you want to build your own decoders)

A zero-ruleset (if you're not looking at actual pattern-matching or IDS functionality) deployment of Suri is surprisingly lightweight and performant, and it's also easy to roll rules for your custom decoders as well if you want to actually alert on certain things instead of just doing straight decoding and dumping to JSON

This might be helpful:

https://github.com/microsoft/ms-zeek

https://zeek.org/

the details here are insanely thin on the how. is this rolled out to every windows 10/11 os in the next security kb? is this a windows feature that can be enabled by enterprise editing only? do I need defender atp license for it? can I fw the data to my detection engine at my enterprise like normal or does this go to a cloud console in azure somewhere first? is this designed to be available to enterprises, or data to improve Microsoft's defender product?

if ms is getting this data, how the hell can they afford the disk space?! my bro implementation was 250GB/h of disk space for 20k endpoints. I can only fathom what all windows agents would generate.

soo many questions around this still

That one is easy, having worked with space-constrained data collection myself (not personal data, it's sensor data so no ethical concerns).

You do a "pre-summarization" of the data on the client side, and just send a report to the mothership. Consequence of this is that it will result in using CPU resources of the client - and that explains why the telemetry service eats so much of it on Windows.

It’s being rolled out as a capability in MDE:

https://techcommunity.microsoft.com/t5/microsoft-defender-fo...

There’s some information here on how Defender approaches this (processing data, nothing specific to Zeek) and which talks about the scale — many petabytes of data and trillions of datapoints.

https://customers.microsoft.com/en-us/story/1540745195786192...

These are paid products/services.

Interestingly you can get access also as small business. Just purchase the Microsoft 365 E5 license. Price is something like 50-70€/month, 12M subscription. This can be convenient, if you have requirements to use this kind of tools from your customers.

And they actually log a lot of stuff. When enabled you can see (on cloud) pretty detailed information on what has happened to workstation.

What I gather and maybe I'm wrong, but it sounds like this is a third party component and not something that is shipped with Windows by default. The post was full of so much BS I had to click off it quickly

Zeek (aka Bro IDS) is awesome for tracking down TCP QUANTUM injection attack incidents: something that Suricata and Snort are still unable to detect.

Zeek script: https://github.com/fox-it/bro-scripts

Presentation: https://old.zeek.org/brocon2015/slides/hu_qi_detection.pdf

Is "QUANTUM" what we're calling TCP hijacking now? From the Fox-IT presentation, it sure seems like this attack is identical to Joncheray's '95 "Simple Active Attack on TCP", and, like, every TCP hijacking tfile written after that. If that's the case, the reason Suricata and Snort might not detect it is that nobody cares; it's an extraordinarily situational, second-order, and low-impact attack.

Quantum insert is actually rather simpler than that attack. Watch for GET request, passively, then inject a response, racing the web server. The real response arrives a millisecond late and gets dropped as duplicate. No need to desync streams.

If you care then you're already running TLS anyway, and they'd only be able to inject garbage. If you don't run TLS then it's already game over anyway.

Only if your still running http without the s, right?

Yes.

Low-level, but yet still a useful tool of and by the determined.

If it's what I think it is, it was extremely useful in the mid-1990s, when nothing was encrypted. Today, it's just one of 100 different ways to deliver a browser clientside exploit, and it's far from the most common or effective. But NSA gave it a stupid name, so people think it's important. It is not.

Most servers do not run a web browser. #headduck

sounds like the most scary kind of surveillance software.. who's it going to provide security for? the host machine, or nation states?

It might be the world's most popular thingamabob but I never heard of it.

> Zeek (formerly Bro)

People that encountered network security probably know the name "Bro" much better. Apparently they rebranded in 2018.

Wikipedia has the following to say about it:

> Dr. Paxson originally named the software "Bro" as a warning regarding George Orwell's Big Brother from the novel Nineteen Eighty-Four. In 2018 the project leadership team decided to rename the software. At LBNL in the 1990s, the developers ran their sensors as a pseudo-user named “zeek”, thereby inspiring the name change in 2018.

Other "fun" fact. I went to a B-Sides conference where one of the higher ups from the Bro Platform talked about how they had to purchase the domain from a fraternity.

Zeek is cool but I always thought Bro was a neat name for a sec product.

It's funny how it was named ironically / in caution, and subsequently changed.

I wonder how long until Palantir rebrands...

Bro is definitely popular, but I wonder where they got the "world’s most popular network security monitoring platform" phrase from.

because it's probably the world's most popular security monitoring platform? it's been around for a long time, it does it's job very well, and it's available in whatever flavor of implementation difficulty you want. from completely free roll yourself, to easy to deploy via push button with thousands of detections already to go for you.

even most of the enterprise solutions sold for network monitoring proudly boast that they use bro under the hood for their stream parsing engine.

> probably

I agree, and I'd like to understand the reason why "probably" disappeared from the press release. If you claim something, you should be able to back it up with some numbers, right? I'm a big fan of Bro and fully agree with the "leading solution" or "everybody is using it" phrases, I just miss the data that would make it the number one.

Everyone uses it. Gov/Military deploy it everywhere and I’d estimate most of the Fortune 1000 does too.

> Everyone uses it.

I hear this phrase about many software platforms. That's why I'd like to see some numbers before someone uses an absolute qualifier. I have no idea how popular is Zeek against OSSEC, Suricata or Snort these days, I'm just wary of claiming something without providing any justification.

I think the real issue here is that things like Snort, Suricata, and Zeek are just not very popular as a class of security thingy anymore. Certainly, for the last 10 years or so, I'd have advised most startups I worked with against deploying anything like them. There was, in the long-long-ago, a huge industry debate about whether detection and response was best done in the network or on the endpoint. That debate has been settled: most places deploy EDR, and few places deploy IPS.

The DoD would beg to differ. EDR is a huge business indeed but often network monitoring is used in conjunction.

Also startups make up almost nothing of the ecosystem. Fortune 1000 and Gov have millions of different departments that have their own requirements.

I’ve never heard of anyone anywhere using such software. In Europe it might even be illegal?

Could your POV be somewhat US-centric?

I’ve been a part of teams that have sold tons of it to European governments and corporations. Anything the US Department of Defense uses has a great chance of being used across Europe. We train Euro militaries on how to use these tools.

In a press release, every product mentioned is the world's most popular product (by some very carefully chosen statistic, of course)

I'm the most popular husband in my house!

Also the least popular.

Also the most mediocre.

A former coworker had a "World's Okayest Dad" mug.

Clearly you've seen my mug collection.

Hey at least that’s deniably dishonest. We are tangibly dishonest when we fund a conference in our sector and win best product in the sector every year!

Same. I've worked in systems engineering for 10+ years and never even heard of it.

I have heard of: Ganglia, nagios, graylog, grafana, science logic and others. And I've used most of them.

None of those are equivalent to zeek. It’s cohort includes snort, suricata, etc.

Yeah so Zeek/Bro might be more of an intrusion detection tool, in the vein of snort or suricata, in that it can monitor network traffic and alert on suspicious patterns. It has some things in common with tcpdump or wireshark also, in that it can interpret common protocols for human inspection.

As part of Windows, I would speculate it serves a function similar to something like Little Snitch on macOS, but that’s just a guess. Maybe they have something different in mind.

Yeah more of a OSS network traffic inspector for security monitoring.

I think it ships with IDS rules nowadays, but we see it generally more for manual impl. We see often fed into Splunk or ELK to feed custom detections, and especially enriching context to simplify investigating alerts by detection tools. Its investment into cross-record correlation IDs make graph-based investigations super effective: you can grab all sessions at an impact period and see them fan out across entities, resources, time, etc!

We mostly see it in sec teams in gov + DIY/code-heavy enterprise. Super popular bc those teams have more time to figure out tuned use of the rich low-level data, maybe budget to store it, and OSS means they can avoid the vendor dance.

CoreLight, who we did a popular webinar with awhile back showing how to enable rich visual hunting & investigation for the data by combining with Graphistry, is the biggest dedicated vendor building hw/sw to make it all more manageable at scale. So seeing in Windows is probably big news for their community..

Zeek can produce a lot of data especially with so many endpoints. If you want an open source low cost way to actually analyze all that data in your own cloud data lake, check out/follow https://github.com/matanolabs/matano.

We're gonna be launching managed support for Zeek soon, where you can just dump Zeek logs in S3 and get out normalized Apache Iceberg tables for all ~43 Zeek logs.

Does it have any options for cloud data lagoons?

I get Zeek (Bro) but what on earth is Microsoft Windows?

> Corelight, the leader in open network detection and response (NDR), today announced the integration of Zeek®, the world’s most popular open source network security monitoring platform,...

How not to write an opener of your press release.

From "the leader", to (r), to "world's most popular" - so much marketing b/s in so few words. It's just awful.

Off-topic curiosity: Zeek was the name of one of the initial high quality (flash) animations [1] I produced before the dot com crash. Afterwards it appeared in the anime oriented TV channel Locomotion [2]. The name was obviously based in the meaning of geek.

[1] http://swain.webframe.org/zeek.html

[2] https://en.wikipedia.org/wiki/Locomotion_%28TV_channel%29

Also the game Zeek the Geek [1] which was a fun part of my childhood

[1] https://archive.org/details/ZeektheGeek_1020

congrats, now MS change the f'ing name, it's mine

use the original name, it's much better:

Microsoft Bro

The company page even makes browsing through logs look cool. Every man's dream job while the teenage hacker guesses the admin's password efortlessly.

It will no doubt be "Defender for networks" or something similar in short order. Nearly everything in the Ms security stack had such a rebrand.

Microsoft Bro got your backdoor all secure

Waiting for it to get cancelled.

Am I just incredibly cynical today or does this read like dense corporate bullshit and lies from start to finish? I have no idea what the software does but it sounds like network-sniffing, privacy-intruding malware being severely sugarcoated? It even starts out with the clichéd line of being “strongly committed to open-source” coming from a Microsoft executive, who (correct me if I'm wrong) haven't open-sourced Windows. Am I the only one who gets dystopian vibes from this?

Zeek is a network protocol parser used to monitor network activity. It parses protocols such as SMTP, DNS, TLS, HTTP, FTP, File, SSH, etc. and collects metadata about that activity.

Here's some of the DNS fields it extracts as it observes the traffic:

https://docs.zeek.org/en/current/scripts/base/protocols/dns/...

Does it decrypt TLS?

If you have the keys, it looks like it can[1][2] (a contingency is that connections must use `TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384`) -- it seems like it's basically the same thing as Wireshark (a network packet sniffer for debugging/forensic purposes).

Not sure why everyone on HN is freaking out about it. It's actually pretty annoying to have to install Wireshark (including their capture driver) every time I need to debug over-the-wire network data.

[1] https://docs.zeek.org/en/current/scripts/policy/protocols/ss...

[2] https://docs.zeek.org/en/current/scripts/base/bif/plugins/Ze...

No. It parses that protocol and logs the metadata observed (time, duration, bytes, source IP, destination IP, ports, cert info, etc.)

Edit: I suppose it could be configured to do that. I've used Zeek in several organizations over the last 15 years and I have never seen it used in that way. However, Zeek running on a client computer (not on a cluster being fed from a 100 Gbit tap or SPAN) would be more scalable. And this announcement is about that.

Here's a nice paper on how Zeek has been configured to monitor fast networks: https://commons.lbl.gov/download/attachments/120063098/100GI...

It’s extremely popular and useful software. Government and corporations use it all over the place.

> I have no idea what the software does but it sounds like network-sniffing, privacy-intruding malware being severely sugarcoated?

It used to be named “Bro” as in “Big Brother”. So yea, it was designed to be very intrusive to privacy concerns.

> Am I just incredibly cynical today or does this read like dense corporate bullshit and lies from start to finish?

It reads like a press release, because it is. Note the "Press Release" banner at top, and the "About *" blurbs at the bottom.

> Am I the only one who gets dystopian vibes from this?

No, also the tool's founder, which is why he named it Bro, as in Big Brother.

I have no idea what the software does but it sounds like network-sniffing, privacy-intruding malware being severely sugarcoated?

AKA 80% of Windows development since 2009?

Why 80% and why since 2009?

Well I'm not going to pretend all of Microsoft's efforts have gone into being progressively more sleazy, just most of them. 2009 is the year Windows 7 released.

What was it about Windows 7 that introduced network-sniffing, privacy-intruding malware?

The idea is that Windows 7 was the last release without network-sniffing, privacy-intruding malware (at least, as it came stock from Microsoft, which it did not when preinstalled on a typical PC).

Remember being excited about the new version of Windows?

For me it seemed that with Windows 11, Microsoft was attempting to recreate the hypetrain they'd built for Windows 95 -- but without the groundbreaking improvements over the previous version of Windows that Windows 95 had. And the result was just desperate and sad.

The biggest difference between 10 and 11 is 11 is much harder to un-fuck.

I'm still excited about new versions of Windows.

...only that now it's more of a panic attack.

Wouldn't the alternative be putting everything behind an airgap and foregoing creature comforts to create parallel infrastructure at great cost? Threat actors don't care about your privacy concerns either.

Ah another thing to assist in the burning of my dick and the flattening of my battery.

May want to rethink your typo while you can still edit your comment.

Reply on news.ycombinator.com