Firefox vs. rthreads | Hacker News Reader

Firefox vs. rthreads

tedunangst.com

220 pointstbirdz48 comments

Open articleView on HN

In other news, the next firefox release is going to have W^X

https://www.mozilla.org/en-US/firefox/46.0beta/releasenotes/

http://jandemooij.nl/blog/2015/12/29/wx-jit-code-enabled-in-...

W^X stands for "writable xor executable":

> What this means is that each page holding JIT code is either executable or writable, never both at the same time.

For anybody, like me, who didn't already know :)

Forgive my ignorance -- what does this achieve?

If an attacker is able to write arbitrary code through to memory, they won't then be able to actually execute that code.

It's a solid security enhancement.

Forgive my naivete, isn't that feature provided by the operating system?

It is provided by the operating system, but you can't just naively enable it by default, especially in an application like Firefox (which has a JIT compiler for Javascript, and therefore needs to be aware of and able to manage which pages are writable and which pages are executable).

Software has to play nice and not request WX pages. If they do, the OS does as its asked.

See also, Theo de Raadt's recent comment on why OSes can't enforce W^X on userland (yet): https://marc.info/?l=openbsd-misc&m=145943630726937&w=2

That you can't have a bug in your code that lets an attacker overwrite data (aka a buffer overflow writes into memory) AND have that part of memory executed. It's either data (which can be overwritten) or code (which can't be overwritten).

It's not a fool proof mitigation for buffer overflow attacks, though. You can still write a bogus return address in a stack overflow and use return oriented programming (ROP) to do exploits with W^X. There are ways to work around address space layout randomization (ASLR) too.

No methodology in Infosec is every 100% prefect. You nest your defenses like Russian Dolls.

Each layer of protection fends off another class of attackers.

An attacker with infinite determination, money, or time will always push past your defenses.

in reality, at some point it's just not cost effective anymore (for the attacker) and other targets become more promising.

None of these mitigations is perfect. But how do you combine a exploit that has to deal with W^X, ASLR, randomized stack gap, stack cookie, randomized location of shared libraries and static programs. Seems pretty hard to me.

> In other news, the next firefox release is going to have W^X

That's nice and all, but if the people behind firefox keep on taking away the features that convince people to use it (Panorama, for starters) then these little accomplishments don't accomplish much.

It's always been Firefox's position that features that don't get used by very many people are better off as add-ons. That way you don't end up with a browser with 1,000 features each of which adds complexity and slows development. The official recommendation is to use this add-on, which was created from the original Firefox Panorama code. http://fasezero.com/addons/

Edit: The page that made the above recommendation https://support.mozilla.org/en-US/kb/tab-groups-removal

That's fine, until they decided to neuter their add-on API.

As long as they're committed to maintaining the feature add-ons under the new API I couldn't care less how it works under the hood.

Regarding Panorama, I was and still am a huge Panorama user and the extension they replaced it with works just as well. Stuff like that is cleaner as an extension anyway (which of course begs the question of why the hell Pocket and Hello aren't extensions).

I appreciate Tedu's musings on technical issues, and how he freely acknowledges when there's a problem on their side. I wonder how Chrome's performance is on OpenBSD?

I worked on the original port of Chrome to Linux. At that time Firefox was much slower than Chrome on Linux. I think one reason was because Firefox relied more on calls into X for drawing (e.g. I believe they used xrender for compositing alpha channels), where Chrome bypassed all of that (just did all its work in a pixel buffer, which is also why we had so many problems with system fonts and system widgets etc.). Unfortunately especially on a platform like Linux where graphics drivers aren't great you're often better off avoiding the system. I imagine OpenBSD is similar -- like if they're using OpenBSD's malloc instead of their faster jemalloc that likely costs a lot.

I also know Chrome also makes a million calls to gettimeofday so it is effectively relying on the Linux vdso performance.

When you are running the same software on multiple OSes side by side (like Chrome on Windows followed immediately on Linux) it sets your expectation for what is acceptable performance. (It's probably worth mentioning here that Firefox on Linux today is much better than used to be.) I appreciate that OpenBSD has a different culture and goals but I feel sorry for them that they don't get a good browser because of it.

At least for me current Firefox on Windows locks up more often than on Linux, but that's most likely due to Windows having more coarse grained locks hanging around in the subsystems than Linux does.

I found this most surprising:

> As you’ll recall from page two of your Building a Multithreaded Kernel textbook, when a high priority thread waits on a lock, it’s supposed to gift its priority to the lock holder to ensure progress is made. We (I) never quite got around to implementing that, and for several years it seemed we just might get away with it. The history of rthreads is pretty much maybe tomorrow, maybe not.

It sounds like OpenBSD has gotten away without having priority inheritance for most (all?) of it's history.

OpenBSD has been fairly niche for most of it's existence. They got SMP support in 2004, but that was 8 years after the project started. They just recently upgraded much of the network stack to use multiple processors more efficiently (or at all?) when processing packets, and that that's one of the core use cases for OpenBSD. In a perfect world, they would have done all this long ago. I'm pretty sure it's just a matter of not enough developers (or not enough developers of enough skill) to accomplish all the things they want to do.

From my point of view it's the "measure before you optimize" mantra. Now with 10Gbit cards a single threaded pf and network stack could no longer keep up with the throughput of the NICs, so something had to be done

The key part here is that OpenBSD take a conservative approach to things, but when they do fix things, it is usually an elegant fix, not a hack to save time now, at a cost of security later.

The key part that I got was that, while OpenBSD (rightly) prioritizes security over performance, all those little performance degradations can add up if you don't pay attention to them.

it runs on some very underpowered machines, so i guess it's not that much of a performance hit.

Regarding the troubles with malloc, Firefox as developed by Mozilla doesn't use system malloc. (It uses bundled jemalloc.)

I don't think this is the case on OpenBSD.

That's my point: They use their own malloc that Mozilla doesn't test with, so slowness there is more attributable to OpenBSD than Firefox.

Mandatorily on all platforms, or just by default? Because of the security & debugging features in OpenBSD's malloc framework[0] I wouldn't be surprised if the OpenBSD port used the system malloc, I'm guessing that's just a compile-time switch away.

[0] http://man.openbsd.org/OpenBSD-current/man5/malloc.conf.5

Perhaps you can check about:buildconfig or about:support for malloc type.

By "as developed by Mozilla", I meant that OpenBSD's non-jemalloc config is their own doing.

>I wouldn't be surprised if the OpenBSD port used the system malloc

I thought so too but then when I looked for such changes, I couldn't find any.

    cvs -qd anoncvs@anoncvs.ca.openbsd.org:/cvs get -P ports/www/mozilla-firefox
    cd ports/www/mozilla-firefox
    grep -Ri malloc .

All that was found was a few irrelevant things which matched because "malloc" was in the name of the referenced files:

    ./patches/CVS/Entries:/patch-js_src_ctypes_libffi_src_dlmalloc_c/1.4/Tue Sep  2 16:43:04 2014//
    ./patches/patch-js_src_ctypes_libffi_src_dlmalloc_c:$OpenBSD: patch-js_src_ctypes_libffi_src_dlmalloc_c,v 1.4 2014/09/02 16:43:04 landry Exp $
    ./patches/patch-js_src_ctypes_libffi_src_dlmalloc_c:--- js/src/ctypes/libffi/src/dlmalloc.c.orig   Wed Jul 23 05:13:14 2014
    ./patches/patch-js_src_ctypes_libffi_src_dlmalloc_c:+++ js/src/ctypes/libffi/src/dlmalloc.c Thu Jul 24 20:47:22 2014

It can be configured to use the system malloc (IIRC running stuff like asan requires this, or at least I have to set a 'no jemalloc' build option), but I'm not sure if any platform ships with this option turned on. OpenBSD sounds like one that might, but I guess I thought they just used the normal Firefox ESR (definitely could be wrong though).

The article mentions that vDSOs have negative security implications. After reading up on what a vDSO is, it is apparently (in Linux) a security fix on an earlier implementation called vsyscall. Can anyone enlighten me what security issues are left in the vDSO system?

I guess there are no known security issues but they are speculating that the vDSO mechanisms may have not-yet-discovered bugs in them.

The idea is just having a special page that is mapped as read-only in userspace and kernel writable. If it's correctly implemented, it should be safe.

I guess they just see every extra interface between kernel and userland as another attack surface and therefore to be avoided.

Right, I don't understand this and I couldn't find an explanation of why OpenBSD has rejected it, at least for gettimeofday(). I can imagine the following:

* The vDSO is mapped at a fixed address in every process, making return-to-libc-style attacks easy. This doesn't seem to be necessary, though, since it's a virtual dynamic shared object, and you can ASLR it just like you can ASLR any other dynamic library. Just tell the program where you put it.

* The vDSO (or at least the vsyscall page, not sure if this is still true) includes a syscall instruction, making it a juicy target for return-to-libc-style attacks. But for gettimeofday() you don't need that. Just include the instruction to read some dedicated read-only page from memory-mapped timer hardware, or run RDTSC, or whatever, and make your ABI such that it permits the vDSO call to fail and regular libc code might have to make a real syscall if it does. Or that the vDSO isn't guaranteed to be present, and is missing if hardware support is missing.

Any further information here?

The vDSO mapping is definitely randomized by ASLR on my system. You can confirm with 'ldd $(which ls)' on your system.

Firefox is pretty terrible on Linux, too. Presumably there's something about their posix threading or X11 integration that's busted, but it's hard to tell.

I've been running Firefox on Linux for 7 years, and I've hardly had any problems.

Try running it on a similar Windows or OS X system. Or try Chrome. Worlds of difference in UI responsiveness.

Make sure you have OpenGL off-main-thread composition enabled. I think it's still disabled by default on X11 systems, because they don't trust the drivers yet. Check about:support for "GPU Accelerated Windows".

about:config → layers.acceleration.force-enabled = true, layers.offmainthreadcomposition.testing.enabled = true, gfx.xrender.enabled = false.

Also, some "beta quality" features: browser.tabs.remote.autostart = true (e10s multiprocess) + layers.async-pan-zoom.enabled = true (exactly what it says, async scrolling like in Chromium, IIRC it requires e10s).

With all these enabled in stable Firefox on FreeBSD, the experience is excellent.

I have neither enabled and Firefox+X11 is the smoothest Firefox on the same machine. May depend on drivers, versions, and config, obviously.

May depend on CPU performance :) On my 3.4GHz 4-core desktop, it is pretty much always smooth. But on my 1.7GHz 2-core laptop, async pan&zoom makes a noticeable difference.

FWIW I'm using it on an old Core2Duo and a mobile Sandy Bridge Dual Core, so nothing fast by today's standards.

For me Firefox runs smoother on Linux+X11, but machines and software versions and configuration can differ, so I believe you when you say it's better on Windows. At least for me Windows Firefox locks up more frequently, perhaps due to Windows' coarse-grained locks in its subsystems.

I know it's anecdotal, but my experience lies with krakensden. I've tried some of the settings lower in the thread, hopefully it'll help.

Reply on news.ycombinator.com