Hacker News Reader

Comment by whalesalad | Hacker News Reader

If I were building a datacenter today I would go with proxmox. It's "just debian" under the hood and can be customized and controlled a multitude of ways (UI, CLI on the box, Terraform, API, etc)

View on HN

I'm using it now, even paying for it, and it works very well but I only have one wish. That they could distribute an image based distro, or a slimmed down appliance ISO. It just seems unnecessary to have an actual OS on each host. Specially since I've been impressed by Talos and Openshift for a while.

https://pve.proxmox.com/wiki/Linux_Container

Not what I meant. That's just how to run LXC containers, which I never use because I find OCI much more convenient.

I meant the hypervisor host OS should be image based atomic/immutable.

Doesn't Proxmox use a separate kernel package compared to Debian? That's kinda annoying because it ends up making the distro a 'Frankendebian' at best. Even using an up-to-date kernel from the stable backports repositories is a lot better than that.

They use a slightly modified Ubuntu kernel (https://github.com/proxmox/pve-kernel), with things like ZFS added. They also really are good about using proper Debian tooling, and so their kernel doesn’t cause any weird dependency issues.

Right now they install proxmox-kerne-6.8.12-6 by default (using pseudo-packages called proxmox-default-kernel and proxmox-kernel-6.8 pointing at it), and offer proxmox-kernel-6.11.0-2 as an opt-in package (by installing proxmox-kernel-6.11)

I’ve been using the latest opt-in kernels on all of my Proxmox nodes for a few years now, and I’ve never had any issues at all with that myself.

> things like ZFS added

That's a big gotcha - ZFS is non-free so of course it cannot be part of Debian proper. Hopefully we'll get feature parity via Btrfs or Bcachefs at some point in the future.

Thats incorrect, it is free software but incompatible with the GPL.

> ZFS is non-free so of course it cannot be part of Debian proper

ZFS is under the CDDL which is a perfectly good free and open-source software license, just some people view it as incompatible with GPL (IANAL, but this is apparently somewhat controversial; see the wikipedia page) so Debian doesn't distribute ZFS .ko files for Linux in binary form. They do, however, have an official package for it[1], just using DKMS to compile it locally.

[0] https://en.wikipedia.org/wiki/Common_Development_and_Distrib...

[1] https://packages.debian.org/sid/zfs-dkms

If some people see it as incompatible, its not perfectly good then, is it.

CDDL is a good license. GPLv2 is a good license. They happen to be (maybe) incompatible. That doesn't make either of them bad. I mean, would you argue that GPLv2 is bad because it's not compatible with CDDL?

I think that you can include it in cddl, but not the other way around.

It certainly has an optimized kernel for its use case. I believe it also includes ZFS by default. I wouldn't be surprised if the Proxmox developers would prefer to upstream these defaults, but they likely would introduce regressions for the common use case that Debian optimizes for.

Ultimately, I use Proxmox as a hardware hypervisor only, so I don't mind that it uses its own kernel. Everything I run is in its own VM, with its own kernel that is setup the way I want.

> If I were building a datacenter today I would go with proxmox.

I use Proxmox as well in a small-ish deployment, but have also heard good things with Xcp-ng.

At a previous job used OpenStack.

I'd be worried how proxmox would scale past a few racks. The bones are all good, but I'm not sure how much scale testing their API layer has had.

Is there a VDI solution that runs on proxmox? I've only found UDS Enterprise.

Or if you'd like to help preventing linux monoculturalization of datacenters, MNX Triton or vanilla SmartOS are very good options too.

The cool thing about proxmox is that it is - again - "just debian" so there is really no vendor lock-in. Yes they do have commercial support/update subscriptions but the community offering is open (https://github.com/proxmox). So I do not worry too much about lock-in or monoculturalization. At the end of the day it is a wrapper around fundamental components of Linux. They do not have any proprietary secret sauce that would F you down the road.

Correction I see now that the projects you reference are Solaris based. I am down with that cause too - but if you are a BSD/Solaris shop expect to do a lot of things on your own. The linux virtualization space is substantially larger (not necessarily suggesting it is better...)

As an aside: Solaris is something you can buy from Oracle, which they forked from OpenSolaris 15 years ago. SmartOS is a distribution of illumos, which also forked from the same code 15 years ago. They have since diverged, in some areas dramatically, so we (the illumos community) don't bill ourselves as being Solaris based.

My impression, strengthened by https://wiki.smartos.org/managing-instances-with-vmamd/#usin... , is that SmartOS preferentially operates at the per-host scale, which is probably a disadvantage in a datacenter setting. (I don't know enough about MNX Triton to comment)

Reply on news.ycombinator.com