Comment by transpute | Hacker News Reader

Comment by transpute | Hacker News Reader

Parent Full thread

> Xen: Large and complicated (by dom0) codebase, discarded for KVM by AMZN

  1. Xen Type-1 hypervisor is smaller than KVM/QEMU.
  2. Xen "dom0" = Linux/FreeBSD/OpenSolaris. KVM/bhyve also need host OS.
  3. AMZN KVM-subset: x86 cpu/mem virt, blk/net via Arm Nitro hardware.
  4. bhyve is Type-2.
  5. Xen has Type-2 (uXen).
  6. Xen dom0/host can be disaggregated (Hyperlaunch), unlike KVM.
  7. pKVM (Arm/Android) is smaller than KVM/Xen.

> The Service Management Facility (SMF) is responsible for the supervision of services under illumos.. a [Linux] robust infrastructure product would likely end up using few if any of the components provided by the systemd project, despite there now being something like a hundred of them. Instead, more traditional components would need to be revived, or thoroughly bespoke software would need to be developed, in order to avoid the technological and political issues with this increasingly dominant force in the Linux ecosystem.

Is this an argument for Illumos over Linux, or for translating SMF to Linux?

> Is this an argument for Illumos over Linux, or for translating SMF to Linux?

I'd certainly like that! I had spent some time working with Solaris a lifetime ago, and ran a good amount of SmartOS infrastructure slightly more recently. I really enjoyed working with SMF. I really do not enjoy working with the systemd sprawl.

I will note the distinction between type-1/type-2 hypervisors never really made sense, and makes even less sense today. http://blog.codemonkey.ws/2007/10/myth-of-type-i-and-type-ii...

I've been using xen in production for at least 18 years, and although there is been some development, it is extremely hard to get actual documentation on how to do things with it.

There is no place documenting how to integrate the Dom0less/Hyperlaunch in a distribution or how to build infrastructure with it, at best you will find a github repo, with the last commit dated 4 years ago, with little to no information on what to do with the code.

> github repo, with the last commit dated 4 years ago

Some preparatory work shipped in Xen 4.19.

Aug 2024 v4 patch series [1] + Feb 2024 repo [2] has recent dev work.

> hard to get actual documentation

Hyperlaunch: this [3] repo looks promising, but it's probably easier to ask for help on xen-devel and/or trenchboot-devel [4]. Upstream acceptance is delayed by competing boot requirements for Arm, x86, RISC-V and Power.

dom0less: ELC2022 slides [5] and video [6].

[1] https://lists.xenproject.org/archives/html/xen-devel/2024-08... [2] https://github.com/FidelisPlatform/xen

[3] https://github.com/apertussolutions/initrd-builder [4] https://groups.google.com/g/trenchboot-devel

[5] https://www.slideshare.net/StefanoStabellini/static-partitio... [6] https://www.youtube.com/watch?v=CiELAJCuHJg

Talking about "technological and political issues" without mentioning any, or without mentioning which components would need to be revived, sounds a lot like FUD unfortunately. Mixing and matching traditional and systemd components is super common, for example Fedora and RHEL use chrony instead of timesyncd, and NetworkManager instead of networkd.

> Talking about "technological and political issues" without mentioning any

I don't know why you think none were mentioned - to name one, they link a GitHub issue created against the systemd repository by a Googler complaining that systemd is inappropriately using Google's NTP servers, which at the time were not a public service, and kindly asking for systemd to stop using them.

This request was refused and the issue was closed and locked.

Behaviour like this from the systemd maintainers can only appear bizarre, childish, and unreasonable to any unprejudiced observer, putting their character and integrity into question and casting doubt on whether they should be trusted with the maintenance of software so integral to at least a reasonably large minority of modern Linux systems.

And people forget that this behavior of systemd devs is present in lots of other core projects of the Linux ecosystem.

Unfortunately this makes modern Linux not reliable.

systemd made time servers a compile-time option and a warn if distros are using the default time servers:

https://github.com/systemd/systemd/pull/554

What's your suggested alternative?

Using pool.ntp.org requires a vendor zone. systemd does not consider itself a vendor, it's the distros shipping systemd which are the vendor and should register and use their own vendor zone.

I don't care about systemd either way, but your own false representation of facts makes your last paragraph apply to your "argument".

> What's your suggested alternative?

That if they do not wish to ship a safe default, they do not ship a default at all.

That would have been my preference too.

The Oxide folks are rather vocal about their distaste for the Linux Foundation. FWIW I think they went with the right choice for them considering they'd rather sign up for maintaining the entire thing themselves than saddling themselves with the baggage of a Linux fork or upstreaming

I read it as "we can sit in this more quiet room where people don't rave about systemd all day long".

But do they? Oxide targets the enterprise, and people there don't care that much about how the underlying OS works. It's been ten years since a RHEL release started using systemd and there has been no exodus to either Windows or Illumos.

I don't mean FUD in a disparaging sense, more like literal fear of the unknown causing people to be excessively cautious. I wouldn't have any problem with Oxide saying "we went for what we know best", there's no need to fake that so much more research went into a decision.

The underlying hyperwiser on oxide isn't exposed to the consumers of the API. Just like on Amazon.

I think arguably the bhyve over KVM was the more fundamental reason, and bhyve doesn't run on linux anyway.

Exactly, then why would they be dragged into systemd-or-not-systemd discussion? If you want to use Linux, use either Debian or the CentOS hyperscaler spin (the one that Meta uses) and call it a day.

I am obviously biased as I am a KVM (and QEMU) developer myself, but I don't see any other plausible reason other than "we know the Illumos userspace best". Founder mode and all that.

As to their choice of hypervisor, to be honest KVM on Illumos was probably not a great idea to begin with, therefore they used bhyve.

FWIW, founder mode didn't exist five years ago when we were getting started! More seriously, though, this document (which I helped write) is an attempt specifically to avoid classic FUD tropes. It's not perfect, but it reflects certainly aspects of my lived experience in trying to get pieces of the Linux ecosystem to work in production settings.

While it's true that I'm a dyed in the wool illumos person, being in the core team and so on, I have Linux desktops, and the occasional Linux system in lab environments. I have been supporting customers with all sorts of environments that I don't get to choose for most of my career, including Linux and Windows systems. At Joyent most of our customers were running hardware virtualised Linux and Windows guests, so it's not like I haven't had a fair amount of exposure. I've even spent several days getting SCO OpenServer to run under our KVM, for a customer, because I apparently make bad life choices!

As for not discussing the social and political stuff in any depth, I felt at the time (and still do today) that so much ink had been split by all manner of folks talking about LKML or systemd project behaviour over the last decade that it was probably a distraction to do anything other than mention it in passing. As I believe I said in the podcast we did about this RFD recently: I'm not sure if this decision would be right for anybody else or not, but I believe it was and is right for us. I'm not trying to sell you, or anybody else, on making the same calls. This is just how we made our decision.

Founder mode existed, it just didn't have a catchy name. And I absolutely believe that it was the right choice for your team, exactly for "founder mode" reasons.

In other words, I don't think that the social or technological reasons in the document were that strong, and that's fine. Rather, my external armchair impression is simply that OS and hypervisor were not something where you were willing to spend precious "risk points", and that's the right thing to do given that you had a lot more places that were an absolute jump in the dark.

I would agree with that. Given the history of the Oxide team, they chose what they viewed was the best technology for THEM, as maintainers. The rest is mostly justification of that.

That's just fine, as long as they're not choosing a clearly inferior long term option. The technically superior solution is not always the right solution for your organization given the priorities and capabilities of your team, and that's just fine! (I have no opinion on KVM vs bhyve, I don't know either deep enough to form one. I'm talking in general.)

Instead people rave about Solaris.

Honestly, SMF is superior to SystemD and it’s ironic it came earlier (and, that shows based on the fact that it uses XML as its configuration language.. ick).

However, two things are an issue:

1) The CDDL license of SMF makes it difficult to use, or at least that’s what I was told when I asked someone why SMF wasn’t ported to Linux in 2009.

2) SystemD is it now. It’s too complicated to replace and software has become hopelessly dependent on its existence, which is what I mentioned was my largest worry with a monoculture and I was routinely dismissed.

So, to answer your question. The argument must be: IllumOS over Linux.

> software has become hopelessly dependent on its existence

With some effort, Devuan has managed to support multiple init systems, at least for the software packaged by Devuan/Debian.

> SMF is superior to SystemD ... [CDDL]

OSS workalike opportunity, as new Devuan init system?

> The argument must be: IllumOS over Linux.

Thanks :)

SMF is OSS. The CDDL is an OSI approved licence. I'm not aware of any reason one couldn't readily ship user mode CDDL software in a Linux distribution; you don't even have the usual (often specious) arguments about linking and derivative works and so on in that case.

>Honestly, SMF is superior to SystemD

Maybe 15 years ago, not by a mile now. systemd surpassed SMF years ago and it's not even close now. No one in their right mind would pick SMF over systemd in 2024.

I regularly pick significantly less featured init systems over systemd whenever it is feasible, because systemd and it's related components have caused some of the largest amounts of work for me over the past decade.

I don't really want to litigate the systemd vs. everything else argument, but as someone that has issues with systemd but is not particularly in love with sysvinit derivatives, I wouldn't mind SMF as an alternative.

The fact that its less opinionated about logging and networking and doesnt ever force any reload of itself are all reasonable reasons to prefer it.

You don’t lose socket activation or supervison. SMF is designed to help work in the event of hardware failure too, which systemd definitely cant handle.

>The fact that its less opinionated about logging

It's simple and straightforward to use any other logging or network configuration system you wish.

>doesnt ever force any reload of itself are all reasonable reasons to prefer it.

It doesn't force reload it self.

>You don’t lose socket activation or supervison.

You don't lose that in systemd either.

>SMF is designed to help work in the event of hardware failure too

So is systemd.

>which systemd definitely cant handle.

It definitely can handle that. One of systemd's core functions is handling hardware events.

The confidence here is breathtaking.

Did AI write this? its completely incorrect.

>The confidence here is breathtaking

Tends to happen when facts are on your side.

>Did AI write this?

"Everything I dislike is AI."

>its completely incorrect.

No it's not.

Yeah, it is. Systemctl reload, journald binary logging being forced on (is an opinion) and so on.

Blisteringly fucking moronic that you double down.

“not losing socket activation” was a reference to the fact that systemd actually gives you that.

>Yeah, it is.

No, it isn't

>Systemctl reload

Yes, that is a feature.

>journald binary logging being forced on

Again, no one is forcing you to use that, this is old FUD.

>Blisteringly fucking moronic

Ad hominem means I'm right and you can't handle it.

>that you double down.

Yes, the facts haven't changed, I double down on the facts.

>“not losing socket activation” was a reference to the fact that systemd actually gives you that.

Good feature.

Reply on news.ycombinator.com