Hacker News Reader

Comment by vlovich123 | Hacker News Reader

Given the multi core, NUMA and Spectre/meltdown reality we’re living in, and the clear benefits of the io_uring approach, why not just have a dedicated core(s) to handle “interrupts” which are nothing more than entries in a shared memory table?

View on HN

This approach works for I/O devices (and for things like network cards the kernel will typically poll them anyway), but I/O isn't the only thing that generates interrupts. For instance, a processor fault (e.g. divide by zero) should be handled immediately and synchronously since the CPU core generating the fault can't do any useful work until the fault is handled.

Is that actually true? Wouldn’t this imply you could launch a DOS attack against cloud providers just generating divisions by zero?

You would only attack yourself. The CPU time you're paying for would be spent processing division by zero exceptions.

Then the CPU isn’t stopping and is moving on doing other work. Meaning the divide by zero could be processed by a background CPU and doesn’t require immediate handling. Same for page faults.

Your instance can still be preempted by the hypervisor.

The hypervisor is just some software that runs on the same CPU. It wouldn't help if the assertion that the CPU can't continue was true.

The CPU can't continue running user code because it jumps to the exception handler. But it can get to the hypervisor at any instruction boundary.

(I write a hypervisor for a living :)).

Incorrect. The "CPU" is stopping and handling the fault. There is no background CPU from your perspective. In a cloud provider you are always in a virtual environment and using a vCPU which is constantly being preempted by the hypervisor.

You cannot DOS a hypervisor just by bogging down your virtualized kernel.

There are a couple x86 corner cases where you can do so, by causing an endless stream of exceptions. But they are not going to happen in a hot path, so the hypervisors force a vmexit on those exceptions and avoid that the processor freezes forever.

The divide by zero in this case is handled by a virtual CPU implemented in the hypervisor, not the actual physical CPU on the server.

A site that renders HTML is more computationally expensive than handling a site that does nothing but division by zero.

There are mechanisms that can intercede, and there are often more than one cpu thread per core.

Faults for divide by zero are a terrible legacy thing, Arm etc do not do this, you test for zero if you want to, otherwise you get a value.

That was just an example, there are many other things the CPU can do that will generate a fault (for example, trying to execute an illegal instruction).

A better example: a page fault for a non-present page.

Could be handled by having the CPU switch to a different process while the kernel CPU faults the data in.

The CPU doesn't know what processes are, that's handled by the OS. So there still needs to be a fault.

You’re thinking about computer architecture as designed today. There’s no reason there isn’t a common data structure defined that the CPU can use to select a backup process, much how it uses page table data structures in main memory to resolve TLB misses.

X86 had the concept of harware assisted context switching. Yet another unsued feature dropped in 64bit mode.

It was slow so operating system devs didn't use it. So it was removed. Probably becouse hardware saved, properly, all registers and software can save only needed few (and sometimes miss something).

In effect: we don't know how secure it was...

But if it was good and Intel removed it then why Intel keeps so many useless crap in ?? Good parts - remove, bad - needed for backward compatibility... Can, finally, someone tell backward compatibility with WHAT ? DOS 4.0 ? Drivers for pre-winonly modems using plain ISA or PCI slots ??

Or maybe just like with EVE Online code (few years ago?) - no one anymore knows how some parts works...

In some cases, yes (if there are other runnable threads on this CPU's queue).

Just to make it explicit for the people having trouble, the mechanism for switching processes in a pre-emptive multitasking system is interrupts.

At university we designed an architecture[1] where you had to test for page not present yourself. It was all about seeing if we could make a simpler architecture where all interrupts could be handled synchronously, so you'd never have to save and restore the pipeline. Also division by zero didn't trap - you had to check before dividing. IIRC the conclusion was it was possible but somewhat tedious to write a compiler for[2], plus you had to have a trusted compiler which is a difficult sell.

[1] But sadly didn't implement it in silicon! FPGAs were much more primitive back then.

[2] TCG in modern qemu has similar concerns in that they also need to worry about when code crosses page boundaries, and they also have a kind of "trusted" compiler (in as much as everything must go through TCG).

Interesting. So what happens if the program does not test for the page? Doesn't the processor have to handle that as an exception of sorts?

This is why you need a trusted compiler. Basically it's "insecure by design" since the whole point of this optimization is to avoid any asynchronous exception so there's no need to implement that in the pipeline. The machine code must be forced somehow to implement these checks.

There have been architectures which have required a trusted compiler (eg. the Burroughs mainframes) or a trusted verifier (the JVM, NaCl). But it certainly brings along a set of problems.

It's unclear from here whether this is even an optimization. It looked a lot more compelling back in the mid 90s.

> where you had to test for page not present yourself.

I would think that implies you need calls “lock this page for me” and “unlock this page for me”, as using a page after getting a “yes” on “is this page present?” is asking for race condition problems.

That would make this conceptually similar to the Mac OS memory manager, with a difference that didn’t use pages but arbitrary sized memory allocations (https://en.wikipedia.org/wiki/Classic_Mac_OS_memory_manageme...)

Yes, but this one is so hard baked into everything that it would kill any form of backwards compatibility.

Or interprocessor interrupts for flushing the TLB, terminating the scheduling quantum, or anything else.

There must be a way to alter a core's instruction pointer from another core or from hardware to support killing processes running untrusted machine code, and to support pre-emptive multithreading without needing to have compilers add a check for preemption on all backward branches and calls.

These features are very worth the hassle of providing this capability (which is known as "IPI"s), and once you have that hardware interrupts become pretty much free to support by using the same capability and the OS/user can then decide whether to dedicate a core, to affinity them to a core, to load balance them among all cores or to disable the interrupts and poll instead.

I was thinking rather than mucking with instruction pointers you would just send a message back to the other CPU saying “pause & switch to context X”. Technically an interrupt but one that can be handled internally within the CPU.

What do you think "switch to context X" does, if not muck with the instruction pointer? Where do you think mucking with the instruction pointer is handled, if not within the CPU?

This is the approach proposed in http://doi.org/10.1109/TPDS.2015.2492542

(preprint: https://science.raphael.poss.name/pub/poss.15.tpds.pdf )

And old systems like Corey OS.

This is essentially the approach taken by

https://www.dpdk.org/ (network) and https://spdk.io/ (storage)

Anything trying to squeeze perf doing IO intensive work should switch to this model (context permitting of course).

Would guess it would be complicated and difficult to update the kernel to support something like that. Not sure Linus would entertain PRs for custom boards that do something like that. Would think it would need to be an industry wide push for that. But just speculation..

We’re talking about the CPU architecture here, not custom one-off ARM boards. Think x86 or ARM not a Qualcomm SoC.

And yes of course. Linus’ opinion would be needed.

Yea, that could probably be an even bigger impediment to trying to implement a separate interrupt processing unit I guess. My grasp of this area is tenuous, didn't even contemplate that. Would a new physical setup be required for something like this? Or would it all be in the cpu architecture?

Reply on news.ycombinator.com