OpenBSD's kernel gets W^X treatment

OpenBSD's kernel virtual/physical memory system, uvm(9), already had the ability to set permissions on mappings. But the recent work done here is still substantial, API's were cleaned up and the W^X policy was applied in the places where memory was being mapped with both permissions (writable, executable).

I'm not overly familiar with all that went into this, but reading the commit logs by both Theo and Mike might help clarify some of it.

http://freshbsd.org/search?project=openbsd&q=file.name%3Aamd...

http://freshbsd.org/search?project=openbsd&q=file.name%3Aker...

http://freshbsd.org/search?project=openbsd&q=file.name%3Aamd...

http://freshbsd.org/search?project=openbsd&q=file.name%3Aker...

Stack smashing attacks are no longer possible, since:

1. ProPolice detects most attempts to clobber the return address.

2. You can't set the return address to memory you control the contents of, such as user input, since that memory is writable and therefore not executable.

3. The remaining way to get code into executable memory is to write a file and have the program mmap(2) it. Address space layout randomisation makes finding this code difficult, even if you have the ability to smash the stack in a way that bypasses ProPolice.

The difficult cases (like the trampoline case mentioned) are usually problematic because they are programmatically writing small functions in machine code, then executing them; basically this requires the discipline to write the function and then immediately flip the page from writable to executable. Implementing a JIT compiler like the JVM would encounter similar difficulties.

It comes from the early x86 processors not having an "no-execute" permission bit for page table entries. In the AMD64 bit architecture, an NX bit was added.

It's fairly rare that you want to write and modify a page somewhat simultaneously, as a high-level goal. Most of the time you're loading some existing executable from disk, and once it's loaded you're executing it, so you can just have the kernel write to it, switch it to executable and nonwritable, and pass it back to the userspace process. Sometimes you have a JIT or a VM doing binary translation, but even there you typically JIT some code once and then call it one or more times, and if you JIT some other code it's a different page. You don't typically intersperse writing code to a page and executing that page. So you just want userspace to be a little rigorous about separating those two steps, and tell the kernel when it's switching between those two, instead of requesting a page that's simultaneously writable and executable.

As another commenter said, one of the reasons this isn't done at the outset is that some architectures (notably x86-32) don't implement W^X in hardware, so there's no pressure to be 100% clean about this. But it's rare that you have code that isn't straightforward, at least conceptually, to rework into W^X compatibility.

One of the more annoying things is fixing up relocations: if you have a call to a function in a dynamic library, and you don't know where that library is going to be loaded until it's loaded, the most obvious way to implement this is to map your program code writable and fill in addresses once you know what they are. Each place where an address needs to be filled in is called a "relocation". So when you load a dynamic library, you loop over all relocations and fill in any addresses for symbols contained in that dynamic library.

There are lots of reasons this is awful; one is that you have to go update every place in the program code. So you indirect that through a thing called the "procedure linkage table" (PLT), which contains a bunch of tiny functions that just go call your real dynamic functions, and you hard-code references to the PLT. The PLT still has to be writable, though. If you don't want that, you make a separate section of the program called the "global offset table" (GOT) that contains addresses, and you have each stub function in the PLT do an indirect function calls to a matching entry in the GOT. So the PLT is executable and doesn't need to be writable, and the GOT is writable and doesn't need to be executable.

If you want to be super paranoid, you resolve all your dynamic libraries at startup and mark the GOT as read-only ("bind now" and read-only relocations aka "relro", respectively), so that nothing is writable. But that's a different discussion from W^X.

(This telling is not very historically accurate about how the PLT and GOT came to exist, but hopefully the explanation of what they do is close enough to correct to convey the general ideas.)

You're correct, the protection is implemented in hardware, but the pages have to be marked appropriately. This message describes a patchset that correctly marks the kernel pages as writable xor executable.

As I understand it, the NX bit is just a new permission bit to say "Don't execute this page". The i386 architecture only had read and write bits, and assumed that read also meant execute.

W^X is a policy that a kernel can choose to implement, that if the W bit is set on a page table entry, so is the NX bit. You need the NX bit to be available in hardware for this to be useful, but hardware support for NX doesn't mean that you have to use it, let alone implement W^X.

This means that amd64 processors are backwards-compatible with kernel and userspace designs that require W|X, even in long (64-bit) mode.

I don't think the first versions of Intels AMD64 chips had the NX bit, but those chips are so old at this point it might be unimportant.

That's right. W^X is a policy that memory is either writable or executable, but not both. OpenBSD uses this model in userspace, now it's being taken a step further and applied to kernel space.

First thought offhand is to have the code be writeable and executable the first time through -- W^X allows for this for JITs and the like -- and then set the page the code lives on to be executable only after this instance. Alternatively, have the once function call a function in memory space that's already set to execute only, to minimize the space attackers can perform shenanigans in.

However, this does kinda ignore one of the big focuses of the OpenBSD project. They tend to shy away from such clever hacks in the name of readability and auditability. While it's definitely a neat way of ensuring your code is only executing once, it becomes a hassle when you have to port it to other platforms. Keep in mind that OpenBSD ports to as many platforms as possible because the subtle quirks of various platforms will often tickle out rare bugs to become more repeatable. In this case, your replacing the once function with a return is dependent on x86, so wouldn't work on the many other platforms that OpenBSD runs in.

JITs are fine as long as you can switch a page from being rw to rx. At least that's my experience writing a JIT in userspace.

For userland, JITs continue work because you can still explicitly request mappings that are both writable and executable. W^X is a default policy.

Exactly yes. A *BSD thing not strictly just openbsd or whatever.

Perhaps it would help to see them "in action" here is a link to the linux emulation layer in freebsd, the MD chapter four specifically describes i386 (this is how you put syscall parameters on, and off, the stack on a i386) and the MI chapter five is a pile of structs that would be used by any emulation layer (NPTL, TLS, the joy of futex'es's (a linux thing that is kind of a mutex cache for speed, sorta kinda), and good luck with the ioctls).

https://www.freebsd.org/doc/en/articles/linux-emulation/inde...

OpenBSD/amd64 does have SMEP/SMAP support, this was committed by Jonathon Gray (jsg@) in 2012, using QEMU. It's still hard to come by in actual hardware.

http://freshbsd.org/search?project=openbsd&q=SMEP&committer=...

http://freshbsd.org/search?project=openbsd&q=SMAP&ommitter=j...

I believe SMAP makes UDEREF obsolete, at least on processors that support it.

OpenBSD does support BPF, but as a character device.