Jitk: A Trustworthy In-Kernel Interpreter Infrastructure | Hacker News Reader

Jitk: A Trustworthy In-Kernel Interpreter Infrastructure

css.csail.mit.edu

66 pointslelf42 comments

The papers linked are from 2013.

Now we know the implications of Spectre make kernel JITs a potential vulnerability. You should not run untrusted code in the same address space with secrets it could steal.

Trusted and trustworthy is the counterpart to proven. Where you assume something is secure instead of proving it to be secure.

You usually prove something with formal verification, and it fails when the assumptions of the underlying model are wrong.

Take the axioms with a grain of salt.

Exactly, the array offset rules are missing important parts. Not just bounds checks, but also offsets larger than a page table need to be forbidden. (Spectre). IMHO arrays should be forbidden at all, replaced by a secure hash map, with a max size.

Way to completely destroy perf by making your memory accesses totally random.

Exactly, People need to feel how bad it is to allow anyone getting user-space access into the kernel. It needs to hurt. But the prime reason is security. solaris and freebsd do have their reasons to disallow arrays in their kernel hooks.

Most Spectre variants are exploitable via specially crafted data, not just code injection. Hell, NetSpectre is remotely exploitable.

Being super ascetic about what code is allowed in the kernel isn't an endgame here. We don't need to take cross domain VMs completely off the table; we just need better ways of describing those security boundaries to the hardware. It's been done before. Some of Azul's secret sauce was additional cheap security modes in the MMU that they leveraged for their GC.

That works, sort of, but how do you even ensure the hardware is actually working? Trust Intel or ARM after what they've shown?

And trying to fully bypass cache and speculation when doing userspace to/from kernel transfers is going to suck. Remapping pages also. And if MMU is untrustworthy, well...

> That works, sort of, but how do you even ensure the hardware is actually working? Trust Intel or ARM after what they've shown?

Which part are you referring to?

> And trying to fully bypass cache and speculation when doing userspace to/from kernel transfers is going to suck. Remapping pages also.

Here's a demonstration of doing it remotely, with specially crafted network packets.

https://arxiv.org/abs/1807.10535

It's not as hard as you're making it seem, the root here is that hardware is currently broken.

> And if MMU is untrustworthy, well...

If the MMU is untrustworthy, it doesn't matter if you allow a cross domain VM or not.

Didn't spectre work cross-address-space ? I mean the whole point of the spectre vulnerability was that one process read kernel memory or another process' memory.

So that would make executing any code anywhere on the same cpu (anything behind the same cache) potentially vulnerable. JIT or not, kernel or not, ...

That's Meltdown, the Intel bug in memory isolation.

Spectre is a CPU cache timing attack to derive data. Cache lines are cleared in between context switches, so it can only affect the same process.

Cache lines aren't cleared on context switches, and there are Spectre variants that are cross address space. There's even a remote Spectre exploit. https://arxiv.org/abs/1807.10535

Oh. I entirely misunderstood, then. I stand corrected.

No, it was not cross address space. Actually the kernel was mapped into the high address space of every process (so it was the same address space) and the kernel mapped all of physical memory. So all of physical memory was mapped into every process' logical address space. This was OK because user processes didn't have the privilege to actually read those kernel pages. But Spectre+Meltdown bypassed the privilege enforcement to speculatively read any location and then bled that information back out via a microarchitectural channel.

The solution was kernel page table isolation. Don't map the kernel and thus all of physical memory into each process. However, this now has the overhead of mapping and unmapping the kernel pages for each context switch.

They're just going to have to fix their broken hardware.

There are Spectre variants that work cross address space. Hell, there's a Spectre variant that works remotely, with carefully crafted network packets.

https://arxiv.org/abs/1807.10535

And don't give me that "oh, we'd have to give up speculation and OoO cores to in order to properly fix Spectre". No, we just need more than two security domains so we can describe to the processor which code should have access to which data. Make Rings 1 & 2 Great Again.

If it delivers on its promise, this could be very useful. I'm no kernel hacker, but any tool that makes working with kernel code easier and safer is a real boon!

There are much better tools. You really shouldn't run user code in kernel space.

Except for example when the user needs to either do something only doable in kernel space or to diagnose and analyze kernel behavior.

I would have loved an option like this for example when working with a power management issue last time (PCI-e surprise removal and hot-plug can be annoying!)... Sure, I can use remote kernel debuggers etc., but those can be sometimes a hassle to set up, not always the most convenient option.

Like when an issue occurs on the other side of the world, and kernel dumps and tracing leads nowhere. Having an extra option for easy injection of code in kernel would be a nice thing to have in the toolbox.

In other words, one-off scenarios. It wouldn't be something you'd release for wider use.

Knee-jerk reactions have been somewhat off-putting lately. All of us should keep in mind other people might have a completely different mindset, problems and needs than our own. We should celebrate new ideas and novel points of view and not to brainlessly use existing dogma or cheap talking points to immediately slam them.

(I've developed (among other things) commercial kernel drivers for Windows. And "for fun" for Linux just to keep my skills updated.)

That's just a side effect of the way Linux puts everything into a giant trusted binary. The GP is correct that we should separate out services (probably to separate sockets until Spectre-like attacks are fixed), even if it's also true that when dealing with Linux-as-it-is-today the solution to everything being in the kernel is to put more stuff in the kernel.

Any sandboxing adds a performance overhead. There is no free lunch in Operating Systems.

A tiny performance overhead is worthwhile in name of security.

For example, I don't remember the last time I cared about disabling bounds checking, even in C++ code (VC++ allows to keep it on).

Tell that the various libc maintainers. They are blocking the C11 Annex K (safe bounds checks) for over a decade now.

And talking about performance With good compilers my secure memcpy_s is actually faster than the glibc or BSD libc memcpy, with compile-time constexpr.

I would blame ISO C, by making Annex K optional, and not caring to improve C's safety in general.

I am quite happy with rust these days. But it still lacks great ‘no allocation’ libraries.

Then you are illusional. None of their safety guarantees are real. Memory, type, concurrency, none. But talking to them fall on deaf ears. Its called hype driven development and very popular amongst HN folks. There exist plenty of real safe languages though.

Then Oracle, Microsoft, Amazon and Google are illusional as well, given their investment into production code Rust.

Any references to support that claim?

The whole point of BPF is that you can't run this code in a separate context _and_ have the code work. The original use case was packet filtering in interrupts because waiting for user space took too long.

If I remember correctly Intel has implemented a user space TCP/IP stack exactly for that.

Which doesn't work if you want to still share an adapter across multiple applications.

I don't see any reason why it can't br made to work between multiple applications while being still faster than traditional kernel based stacks.

The whole point of DPDK is to avoid the context switches by dedicating the adapter to a single application, and letting the application take over management. Once you have multiplexing, you're right back to where you started. There has been a secure multiplexing scheme based on packet buffers... the Berkeley Packet Filter (BPF), ie. what this paper is talking about as prior art.

No, you can simply have memory mapped ring buffers between processes and all of the stuff not required for the specific application cut away. You don't need to have any more context switches that way.

No need to have a traditional socket API while still being able to do access it from multiple applications.

I have no interest to create such beast, but I'd be truly shocked if it couldn't at least beat a generic kernel based stack.

It'll lose some performance compared to the single application approach, but there might still be a niche for this kind of way.

Sure, you'll need to copy memory, but the data should be almost always in L3 cache anyways.

Once you're doing copies (in your example two copies!) it's game over.

My intention was however to only have one copy (other than initial DMA to (hopefully) directly L3 cache).

Yeah, it hurts if it's 400 Gbps ethernet. L3 bandwidth is like 50-90 GB/s. X86 just doesn't have enough bandwidth even to the caches! Better have pretty high CPU frequency, as I think L3 gets faster in proportion (but not completely sure). 200 Gbps should be somewhat fine.

Also pretty bad if the data travels over a QPI link... better have both processes in same NUMA region. And that ethernet PCI-e adapter... :-)

Regardless, I do think it'd still work way faster than anything a reasonably general kernel stack could do. Might be a reasonable compromise when process & permission isolation is required.

Bpf can handle 60 million packets per second, adding a user/kernel context switch will kill that by a factor of 1000x. So while your code may not care about it, there are definitely low latency applications where milliseconds equate to dollars.

Some high integrity OSes do that full in user space, while running Linux on the side as it isn't certified for such scenarios.

> The GP is correct that we should separate out services...

That's why I said "It wouldn't be something you'd release for wider use.".

I wanted to point out there are use cases where a full-fledged kernel module would be an overkill, and some safety would be desirable.

Not safety and security from others, but from myself, so that small dumb mistakes are less likely to cause kernel crashes. For research, debugging, curiosity, and so on. Not necessarily for production use!

Although, now that I think about it, it would be nice to be able to sandbox trusted code as a part of a kernel module as well. Writing kernel code is hard work, because it needs to be truly correct.

There's often code in kernel drivers that's not performance critical, but must be done in kernel mode and must be crash-free and secure to execute. So why not take advantage of sandboxing for additional safety?

That is what Windows 10 introduced with its secure kernel hypervisor.

https://docs.microsoft.com/en-us/windows-hardware/design/dev...

What would this sandboxing achieve? If you mean kernel internally using multiple protection rings, tricky to do in Linux, because the API in that either has to be tight or it's still not too secure. E.g. if you just take RCU or softirqd into near userspace, as done in RT patchset, but then introduce context switching overhead, life is going to be relatively ok, but you've covered only relatively small parts of the drivers. And RT breaks drivers without even introducing more overhead. It would take a major overhaul.

How does this compare with eBPF?

It is BPF. It's one of the pavers in the road that got us to the current eBPF implementation today.

Reply on news.ycombinator.com