Comment by cmrdporcupine | Hacker News Reader

Comment by cmrdporcupine | Hacker News Reader

Parent Full thread

> Why? The solution thus far has been to turn off what the kernel does, and, do those things in userspace, not move everything in the kernel? Where are these performance gains to be had?

There's all sorts of jankin' about trying to squeeze ounces of performance out of the kernel's page management, specifically for buffer pools.

e.g. https://www.cs.cit.tum.de/fileadmin/w00cfj/dis/_my_direct_up...

Page management isn't really a thing we can do well "in user space". And the kernel has strong ideas about how this stuff works, which work very well in the general case. But a DB (or other system level things like garbage collectors, etc) are special cases, often with special needs.

LeanStore, Umbra, etc. do tricks with VMM overcommit and the like to fiddle around with this, and the above paper even proposes custom kernel modules for the purpose (There's a github repo associated, I'd have to go look).

And then, further, a DB goes and basically implements its own equivalent of a filesystem, managing its own storage. Often fighting with the OS about the semantics of fsync/durability, etc.

I don't think it's an unreasonable mental leap for people to start thinking: "I'm by necessity [cuz cloud] in a VM. Now I'm inside an OS in a VM, and the OS is sometimes getting in my way, and I'm doing things to get around the OS... Why?"

> Page management isn't really a thing we can do well "in user space".

But it is the thing most high performance OLTP DBMSs, most of us are aware of, do? I'm also not sure your cite is relevant here. Or it is at least niche. The comparison is made to LeanStore, which is AFAICT is not feature complete, and a research prototype?

Your cite does not describe a unikernel use case, but instead in kernel helper modules. Your cite is about leveraging in kernel virtual memory for the DB buffer cache, and thus one wonders how sophisticated the VM subsystem is in most unikernels? That is -- how is this argument for unikernels? Seems as though your cite is making the opposite argument -- for a more complex relationship between the DB and the kernel, not a pared down one.

> And then, further, a DB goes and basically implements its own equivalent of a filesystem, managing its own storage. Often fighting with the OS about the semantics of fsync/durability, etc.

The fights you're describing what have thus far been the problems of ceding control of the buffer cache to the kernel, via mmap, especially re: transactional safety.

If your argument is kernels may need a bottom up redesign to make ideas like this work, I suppose that makes sense. However, again, I'm not sure that makes unikernels more of an answer here than anywhere else, though.

> I don't think it's an unreasonable mental leap for people to start thinking: "I'm by necessity [cuz cloud] in a VM. Now I'm inside an OS in a VM, and the OS is sometimes getting in my way, and I'm doing things to get around the OS... Why?"

I think that's a fair thought to have, but the problem is how it actually works in practice. As in, less code seems really enticing, the problem is what abstractions are you throwing away. If the abstraction is less memory protection, maybe this is not a good tradeoff in practice.

Reply on news.ycombinator.com