McKernel – A lightweight multi-kernel OS for high-performance computing | Hacker News Reader

McKernel – A lightweight multi-kernel OS for high-performance computing

www-sys-aics.riken.jp

125 pointsrbanffy30 comments

> On the other hand, a new class of more dynamic and complex applications are also on the horizon, with an increasing demand for application constructs such as in-situ analysis, workflows, elaborate monitoring and performance tools. This complexity relies not only on rich features of POSIX, but also on the Linux APIs (such as the /proc, /sys filesystems, etc.) in particular.

If I were to design a new class of a thing to manage complexity efficiently today I would rather choose to rely on something Plan9-like or entirely new rather than the legacy "rich features of POSIX".

But you don't get the choice, given the applications. It's not clear to me how Plan9 would help with relevant problems; it was tried on Blue Gene at one stage. If I got the choice of something different, it would likely be capability-, microkernel-based, which might actually help with the jitter problem.

there is an L4 based effort for HPC at TU Dresden called FFMK. the reality is that for HPC you still need high-performance implementation of fast-path system calls (e.g., memory and process management) and the traditional micro-kernel architecture is not suitable for that. as for POSIX, you don't have a choice, people who write apps are used to POSIX and Linux.

Hi, I'm another person involved with McKernel from early on. The name actually originates from Many-core Kernel (intended for many-core CPUs). In addition, a clarification regarding standalone LWKs, mainly two reasons why they "failed": 1.) incompatibility with POSIX/Linux, 2.) lack of device drivers and thus their lack of applicability to different platforms. IBM's CNK (still running on the BG/Q) is a prime example. Plenty of POSIX calls are not supported and it only runs on IBM's hardware. The multi-kernel approach solves these issues by providing basically full Linux compatibility through service offloading and easy deployment thanks to direct reuse of Linux device drivers.

Site looks down right now, alternative link without css (looks bad, but better than nothing until news link comes back up):

https://www.sys.r-ccs.riken.jp/ResearchTopics/os/mckernel/

Also, not mentionned there but the code is on github:

https://github.com/RIKEN-SysSoft/mckernel

https://github.com/RIKEN-SysSoft/ihk

I'm more curious about this Interface for Heterogeneous Kernels (IHK) that makes running Linux next to McKernel possible.

It sounds like something like virtual partitions (vPars) from old HP Superdomes, which were pretty neat.

It's basically a few kernel modules and utilities that help with memory/cpu management (take cores offlines, reserve memory etc) and provide message passing from linux to whatever you want to run on the other codes with a queue messaging system (ikc) -- it's used by mckernel but should theorically work with anything else, would love to know if anyone else uses it (disclaimer: I'm working on ihk/mckernel this year)

I would think there's likely to be some security implications with this that aren't acceptable at a lot of HPC centers (at least in the US)

Can you be more specific? If previous LWK approaches were OK, what is different about this?

I think the LWK concept itself is ok, but implementation-wise there need to be some auditing done. This is in a phase going from mostly being a research project to getting some attention and setup for production use on some HPC sites so I'm sure that'll get done in due time.

I doubt the number of possible privilege escalations would be much different from on your average compute node. There might actually be fewer if syscalls could be sanitized before reaching Linux.

Does anyone ever build something and then advertise it as heavyweight?

Yes. Very often. They usually use such synonyms as "capable" or "sophisticated."

You will also see words like "feature-rich", or "all-in-one solution".

I often hear the term “batteries included,” used to describe tech that comes with lots of features/stuff out of the box, but basically if something is heavyweight, what is advertised are its feautures, not its size.

Wouldn't popcorn be a better name?

"McKernel" is great, it both somehow refers to the architecture, to the fact that it's a kernel, and is unique and googleable.

"Popcorn" is generic and makes it hard to search for the project, and easy to clash with another project of the same name.

I wish people invented good unique name for projects, instead of trying to make a (b)ackronym or a subtle reference using a generic word.

This does kinda sound like something off the Dollar Menu, though.

I thought of it as a joke name like Parsey McParseface: https://deepai.org/machine-learning-model/parseymcparseface

Only to the American audience. Doubtful that it evokes the same association in Japan (www-sys-aics.riken.jp).

McDonald's is in Japan.

But in Japan, is the culture there to associated all the Mc's with McDonald's, like in America? Do they even have the "Dollar" menu (Yen menu?).

Anyways, I doubt naming McKernel had any thoughts of McDonald's and the implication of the cheap dollar menu.

They have おてごろマック, which is ¥200: http://www.mcdonalds.co.jp/menu/otegoro/

They'd have called it the Kernel Royale.

Kernel Royale doesn't sound cheap to me, in fact, sounds regal =P

https://www.youtube.com/watch?v=6Pkq_eBHXJ4&feature=youtu.be...

Royale with cheese!

Popcorn Linux is a thing, and it's a multi-kernel OS too: http://popcornlinux.org (I would know, I've been working on it for a few years).

There was even such a thing as "MkLinux", once upon a time! It was the first Linux variant I ever used.

Reply on news.ycombinator.com