Hacker News Reader

Comment by mconley | Hacker News Reader

TL;DR: Windows Defender had a bug that made certain system calls expensive on CPU cycles when Defender's Real-time Protection feature is enabled. After discovery, Mozilla reported this issue to Microsoft. Microsoft is releasing a patch that should result in lower CPU usage when using Firefox on sites like YouTube (a ~75% CPU usage reduction was noted when browsing YouTube in Firefox with the fixed version of Defender).

It seems like the HN submission form truncated the # from the end of the URL I linked to, which linked to the relevant comment. I'll try that here:

https://bugzilla.mozilla.org/show_bug.cgi?id=1441918#c82

and

https://bugzilla.mozilla.org/show_bug.cgi?id=1441918#c91

View on HN

It's not just mozilla, been working defender issues for the last few years on thousands of windows vm's. Mostly due to the enabling the more intensive heuristic real time engine and they have different code bases depending on versions installed on different windows builds, and patching does seem to trigger it. For months we had issues where we couldnt log into some vm's due to high cpu for defender, and had to bounce the vm and apply a temp defender fix.

I think its a growing issue, as they mature/migrate their older code base, issues become less frequent.

I have malwarebytes premium and defender CPU usage is nearly 100% at times bringin Firefox to a halt. Chrome works fine..I've been blaming Firefox so far.

In my experience (as a former Firefox dev), antivirus / antimalware software are really poorly behaved. They tend to:

- require admin rights (which means that if they have vulnerabilities, it can take control of the entire machine, even if Firefox itself is sanboxed);

- monkey-patch the Firefox executable in memory, which works (when it does) as long as the version of the software tracks closely the version of Firefox, which may or may not be the case;

- ... and also decreases the memory-safety of Firefox, which makes it easier to pwn;

- ... and also makes the crash reports unreliable;

- install encryption certificates that are actually less trustworthy than Mozilla's, hence decreasing the security of https;

- block Firefox and add-on security updates, also decreasing security;

- install privileged add-ons, many of which are easy to exploit from any webpage;

- ...

Part of the work on Crash Scene Investigations was attempting to determine whether the crash was in Firefox or in code or in some bogus foreign code. Depressingly often, it was the latter.

In your case, it's entirely possible that malwarebytes was simply untested on Firefox.

> - monkey-patch the Firefox executable in memory, which works (when it does) as long as the version of the software tracks closely the version of Firefox, which may or may not be the case;

This one was a frustratingly common cause of crashes when I worked in gamedev. So many crashes would end up being some overlay or antivirus monkeying about with memory.

I had always assumed that one application could not touch the memory of another application. Does running as Admin allow breaking this boundary?

This is wrong, on Windows there are system calls to access memory of other process and on Linux you can do it using debugging. Also on Windows there is a tradition to inject libraries into other processes, create threads in processes etc.

On Linux, ptrace permissions can be restricted [0] and some distributions do this by default.

Whether this provides any meaningful security is questionable unless you pair it with filesystem isolation to prevent malicious programs from modifying config files / bashrc / etc. Meanwhile it does make legit uses of ptrace more annoying.

[0] https://www.kernel.org/doc/Documentation/security/Yama.txt

Yes. However, I think parent process can gain access to child process memory without admin rights.

All that's generally required is being the same user at the same or higher integrity.

Yes, in general on Windows processes with higher privilege levels can get access to read/write another processes memory, or even inject code into them. And even Admin-level processes can still be broken into by something running as a service with even more elevated privileges like NT AUTHORITY\SYSTEM.

This has long been a leaky part of Windows security. If your malware can get its code running inside a highly privileged service or process, it can do more or less whatever it wants to the rest of the system. But even when not used for nefarious purposes, it is still an extremely dangerous capability in that it can be very easy to create problems .

Anything you run as your user can be accessed.

cheatengine, wemod, and so on would not be able to work if this were the case. Thankfully those all work, at least up to windows 10!

Or userland debuggers.

They work just fine in windows 11 so far.

By default, any application's memory can be read and written to by other processes running as the same user, as far as I know. The way to deal with this is to set process security descriptors, but admin can still bypass this. There are protected processes, and protected processes light, but those are not used by most software (mainly anti-malware afaik.)

https://learn.microsoft.com/en-us/windows/win32/procthread/p...

There are protected processes, and protected processes light, but those are not used by most software (mainly anti-malware afaik.)

...and DRM.

Although that was definitely the intent, I actually don't know about specific things that use it. I'd love to hear what actually uses it. (I don't think Widevine l3 does, for example.)

I seem to recollect that iTunes did, but maybe that was just on OSX.

This is an EXTREMELY common pattern in the world of Windows... Especially with antivirus

How you debug then?

This almost reads like Defender makes machines less secure on purpose.

Makes me wonder: Does windows Defender just double as another deliberate NSA backdoor?

Why would Microsoft need to put a NSA backdoor specifically into Defender when it could put it anywhere else into Windows with their monthly patch? It doesn't make sense to single out Defender.

The same is valid for Apple, Google, and every other US company.

Pretty sure Defender is one of the few anti malware/edr that doesn’t need to do this, because it’s so tied to the platform. 3rd party antimalware and EDR are much more likely to inject hooks and dlls into other processes

> - install encryption certificates that are actually less trustworthy than Mozilla's, hence decreasing the security of https;

Given that in many industries insurances and, in some cases like banking, the law requires companies to monitor HTTPS traffic of browsers for compliance, it might be better if browsers had a dedicated filter / monitor API.

WebExtensions definitely have such an API. That's how AdBlock, uBlock, etc. work.

> Part of the work on Crash Scene Investigations was attempting to determine whether the crash was in Firefox or in code or in some bogus foreign code. Depressingly often, it was the latter.

A shockingly large number of crashes and performance issues in PC gaming are related to poorly behaved overlay programs and overclocking tools like RivaTuner, Overwolf, and the Discord Overlay. I'd well believe your points.

Did you report this on Bugzilla? Even if it's not Firefox's fault if you don't report it no-one knows about the issue.

Recent versions of Firefox allow you to block some stuff like that: https://support.mozilla.org/en-US/kb/identify-problems-third...

Though it's possible they use different code injection tricks to make blocking impossible. (You can't block Defender from listening to events for example)

I am on Windows 10, Malwarebytes premium and using Firefox Nightly on Youtube right now and it is using miniscule CPU and has so for a long time. On a i7 4790k desktop machine.

Firefox itself is at 4-5% and the whole machine is at 14%

Normal Firefox was also fine last I used it.

I'm curious how much excess energy has been consumed, and won't be consumed any longer, as a result of this improvement - even just limited to reduced CPU usage on Windows machines using Firefox to watch Youtube.

I love thinking about the impacts of tiny improvements at scale like this, might do some napkin math on it later and see if I can come up with something in the right order of magnitude.

firefox browser share is teeny tiny these days

Teeny tiny multiplied by 7 Billion by 365 days per year by 24 hours per day by a fraction of a kW does add up.

7B people are not watching youtube on Firefox 24/7 365 days a year.

But at any given moment someone is.

Correct. Some teeny tiny fraction of market share is. For the conceptual calculation, I refer you to my earlier comment.

*8 Billion sir

https://www.worldometers.info/world-population/

Oh no! My mistake.

It's 200 million people, with 80% on Windows.

https://data.firefox.com/dashboard/user-activity

https://data.firefox.com/dashboard/hardware

Next: Canadian cars and their daytime running lights.

Running lights during daytime seems to reduce crashes by about 5-10%, and crashes consume a lot of energy. Depending on crash severity there's at a minimum the wasted time for all involved parties and frequently the necessity for repairs (including the production of replacement parts, paint etc), and at the high end the involvement of emergency personnel and their vehicles, hospital beds, doctors, the production of entire new cars as replacement for totaled ones, etc.

I'm not so sure that running lights isn't a net positive, especially with the introduction of LED lights.

On the other hand, crashes also kill people who will then stop be using any energy.

Even LED DLRs?

Next: internal combustion engines doing more heat than torque.

That's a feature in winter. Portable propane heaters for cars are a thing. I think they'll sell a lot for EV cars.

I don’t know. It’s not a thing in Norway and we have plenty of gaz, electric cars, and cold weather.

Now calculate the man years lost to fixing strings represented as exponents in excel.

> “Windows Defender had a bug that made certain system calls expensive”

It also has a bug(?) which makes method calls 100x slower in PowerShell 7:

https://github.com/PowerShell/PowerShell/issues/19431

Note that this issue is not exclusive to MS Defender, but likely all Windows AV products to varying degrees:

> > I would also like to add that this high CPU usage issue while using Firefox is not exclusive to Microsoft Defender. It's an issue for Norton's AV products also and should be the same for Symantec Endpoint products too.

> > So, you should also test them.

> It is true that we should analyze the situation with other AV vendors, however, given the numbers shared above, and given how relevant it is to keep track of memory protection changes in order to detect malicious behavior, it is very likely that the explanation for Windows Defender also applies (at least in part) to other AV vendors.

Can we get edit on the title?

I've seen some really weird performance behavior from Defender and I just keep it disabled on my desktop device now. I'm not surprised to see it affecting Firefox like this. Defender's dropped all the way to the bottom of the list in effectiveness anyway, so I don't feel it's a big loss.

Ok, I've put that back in the URL above. Thanks.

The biggest surprise for me was Microsoft actually fixing it.

Is that because you don't expect programmers in general to fix their bugs? Or do you think Microsoft in particular don't care about their products?

Because once a corporation grows larger than some singularity threshold, there seems to be a bug event horizon where all bug reports just disappear.

Send a bug report to a five-person software company, their lead dev contacts you the same day and has a patched version ready to go in a week. Send a bug report to Microsoft / Citrix / Apple / etc, and you'll never hear back.

They said that Microsoft was already in progress of changing the offending code before Mozilla report.

Well, also Firefox is making an excessive number of calls to that slow system call compared to other browsers (Chrome, Edge).

My understanding is that until recently (January), V8 (inside Chrome & Edge) made a similar number of calls. The main use is making it so that JIT-generated code is not writable while it is executing. It's an important security measure. V8 switched to a more recent mechanism (memory protection keys) that have been gradually getting support from the various OSes. But IIUC, they switched off the mprotect/VirtualProtect calls unconditionally, and added in the protection key stuff only where supported, which suggests that they left some configurations without any protection at all. SpiderMonkey (in Firefox) has not yet switched to the cheaper mechanism.

I may have some of the details wrong.

https://source.chromium.org/chromium/_/chromium/v8/v8.git/+/...

pkeys are hardware-specific as far as I am aware, and at least last time I tried them didn't work on hardware as recent as zen 1.

Well, it was fast system call until MS added AV hook to it.

My comment was only intended to add missing information to the TLDR (since this fact is important in the linked thread) not to say that Firefox is at fault.

Now that you raised it however, even if the system call used to be fast, Firefox is making an extremely high number of calls to that sytem call, and there's always going to be some overhead to that. There are almost certainly ways that Firefox could reduce the number of calls it needs to make.

What determines "excessive"? In this case it's for a security feature that Chrome lacks(!).

Reply on news.ycombinator.com