Hacker News Reader

Comment by ams6110 | Hacker News Reader

Kernel code is not my area. Is this substantially different from how Linux is addressing the issue?

I work on the Linux KPTI/KAISER code (and work at Intel).

The high-level description, along with the set of hardware things that get mapped into both kernel/app page tables are basically identical. Creating that list of things was painful, at least for me while I was getting KAISER working. If I were in their position I definitely would have leveraged the list of things that we found in the Linux work. I hope they were able to do that.

Some minor differences in the implementations: In Linux, we allocate the two top-level page tables next to each other so we can just flip a bit to switch, and in the OpenBSD commit they seem to be allocating them separately and then storing the two pointers in a per-cpu data structure. Both are fine ways to do it.

I don't see anything like the Linux "espfix" mechanism. Guess there isn't one.

The BTS/PEBS buffers seem missing. This hardware may just not be supported.

Having separate page tables for user space vs kernel space seems like the right thing to do from the beginning -- I was surprised that they were shared.

Could you elaborate on "we allocate the two top-level page tables next to each other so we can just flip a bit to switch"?

Other architectures (like SPARC) have had split PTs for like, always. (And they're not optional, either)

Sharing an address space is nice. Programs pass lots of pointers into the kernel (think read()/write()). For the kernel to get at that data, if the kernel and userspace share an address space you can just use the pointer. If userspace has a truly separate set of page tables, you have to do some relatively crazy (and slow) stuff to go and access the data from inside the kernel. It's much harder than just accessing a pointer.

We can still do this with KPTI because despite having two page tables, the kernel still continues to map all of userspace.

BTW, the bit flipping is pretty much

// Allocate 8k which is also 8k aligned: pgd = alloc_pages(PAGE_SIZE*2);

That allocates something which might go from (physical address) 0x12340000->12360000. The first top-level pagetable is at 0x12340000 (for the kernel) and the second is at 0x12341000 (for running userspace).

So, when you leave the kernel you just set a bit in your CR3 register (well, you do it in assembly, but here it is logically in C):

write_cr3(read_cr3() | 0x1000); // 0x12340000 -> 0x12341000

When you come back in to the kernel you clear the bit:

write_cr3(read_cr3() & ~0x1000); // 0x12341000 -> 0x12340000

The implementation is of course different, but the workaround is the same, separating the user/kernel pages tables. One key difference is that OpenBSD doesn't have to deal with 32-bit binary compatibility [0] and such, and also has made considerable effort to minimize the amount of kernel pages needed, AFAICT 6+(2*ncpus)

[0] https://marc.info/?l=openbsd-misc&m=148926149318522&w=2

No, but note that OpenBSD didn't learn about the issue until the public announcement. Linux developers had months to work on addressing the issue.

In Theo we trust

In this case not really. The OpenBSD impl. looks very costly and is still incomplete. You may wonder why they didn't add a benchmark number to the commit. I suspect there's a lot of pagetable cache trashing going on, with two complete separate pages.

See https://news.ycombinator.com/item?id=16441341

While there are obvious disadvantages to not being told early, could there have been some advantages? For example, Linux and Windows mitigations already worked out and ready as a reference?

(I'm not saying those advantages might outweigh the disadvantages, just speculating on the pros and cons!)

If you're downvoting, I'd love to know why.

I asked an honest question, made no statements to which I can imagine people disagreeing with, and went out of my way to make sure it couldn't be taken in a bad way...

I mean... if you want to be really pedantic, there are technically some silver linings. But the disadvantages outweigh that stretch of a silver lining by so many orders of magnitude that it's almost insulting.

It would be like telling a mugging victim: "Hey, at least now you can justify buying a new purse!"

The basic of concept of separate page tables for user land and kernel is well known. It's how sparc64 is implemented from the very beginning. But knowing this is the objective is almost meaningless to actually accomplishing it. The devil is in the details, of which there are many, and they aren't necessarily shared.

Was any reasoning made public for this decision? It seems bizarre to have left the *BSD projects in the dark.

I have heard it mentioned that the core Linux kernel developers didn't get told months ahead - only some people at Red Hat specifically. Anyone know the truth of that?

Reply on news.ycombinator.com