Hacker News Reader

Comment by nubinetwork | Hacker News Reader

> Tesco is also dealing with migration challenges related to data security because its new, unnamed virtualization software is incompatible with the Veeam and Zerto products it uses.

What is a VMware alternative, that isn't compatible with backup software? I'm guessing it's not nutanix?

View on HN

I'm having flashbacks from the late 90s/ early 00s when your company would hire a "Linux guy" that would force a large scale migration to some open source stack no one heard of, then only later worry about if any existing applications worked.

Currently in Finland, a major public health provider is moving to chromebooks. By the end of 2026. They won’t even have the test environments ready before Q3 2026.

Interesting times.

I've been hearing that HPE are on a push lately with larger enterprises trying to encroach on VMWare during their pricing changes, might be them.

HPE's VMWare alternative is Morpheus, and it supports both Veeam and Zerto. So it's probably not them.

Probably Proxmox. Veeam support is relatively new.

Proxmox for 40k vm would be surprising also veeam support Proxmox.

I'd would assume that this is not a monolithic cluster of 40k vm's but at least tens of clusters. Which puts it in the realm of capabilities of Proxmox.

Before my vacation we (3 colleagues and myself) completedan 8 months long migration (coordination with stakeholders is longer and more complex than migrating a 192TB VM !!!) to 6 proxmox clusters so 20 to 40 clusters for 40k is certainly possible but imo it would be unwieldy.

Would you do it again (ProxMox)?

The FC multipathing was a learning experience and the manual workload placement requires good metrics on your workloads. The built-in ceph is decently configured and more performant than FC if you have 100GBS mellanox NICs and adequate quantity of ram. Veeam integration is serviceable but it's not as mature and polished as the integration available on vmware.

Having tried azure local before (it seems magical but the more you use it the worst it gets, update failed for no apparent reason on only some supposedlyidentical nodes, the sdn was atrocious to deploy and was manageable from wac only), I would recommend proxmox over it anyday.

If you don't have linux expertise on hand and have traditional FC based storage, I would recommend something else, probably nutanix if your budget is big enough.

And if you had to do it one a normal human setup of three hosts and 15 total VMs, no fiber, no san, no special nics or anything like that, just tied in to an ordinary Veeam with the ssd storage on an immutable Linux machine elsewhere?

I would choose promox or ovirt (oracle virtualiztion manager is pretty nice, I use it because the licence of the only oracle database of our institution requires it, that and a specialc cores pinning configuration, to avoid licensing every cores in all clusters, https://houseofbrick.com/wp-content/uploads/2015/03/Oracle-P... ) but it's oracle so we went with proxmox.

> 192TB VM

Why? Honest question, what leads to that Kind of size and why cant it use NAS shares or SAN disks for most of that data? Kudos on the migration!

I wish I knew ... at least it was a span volume so we could use proxmox support for vmdk and achieve about 30 minutes of planned downtime but a week of storage vmotion followed by another week of the proxmox equivalent.

When I get back to work I will finish the samba configuration of the ceph cluster front-ends to replace those elephants.

I’d guess thousands of clusters. They have over 3k retail stores in the UK, so that could be a 2-3 node cluster in every one.

I’ve worked with a few major US grocers on very similar projects (some hardware only refreshes and one VMware to HyperV/Azure Local migration).

> Probably Proxmox. Veeam support is relatively new.

As a sysadmin of Proxmox, I do not see how it can scale to 40k VMs. The Proxmox folks themselves have seen "~24" nodes in a cluster (theoretical support is higher), so you'd probably need a lot of clusters for 40k:

* https://forum.proxmox.com/threads/proxmox-with-48-nodes.1746...

For such a size (and sticking strickly with open source), XCP-ng could be an option, or OpenStack. In the closed source space, Nutanix.

As of 2021, CERN had 35k instances/VMs in their OpenStack implementation:

* https://superuser.openinfra.org/articles/scaling-bare-metal-...

Nutanix has served us well over the last 8 or so years.

How has pricing been? I’ve heard their renewals can get expensive.

Have lots of customer who run it and would echo your same positive review.

I don't love their pricing but it's not brutal like others I've seen. We run 3 clusters - one of which is an old legacy cluster that was our original and despite Nutanix not supporting it officially (meaning we don't pay for support on it), their support has still been helpful with it and like the other 2 clusters, uptime/stability has been rock solid.

Prism Central has definitely gotten better with the UI since the earlier days. I still prefer Prism Element in some cases, but overall it all works pretty well.

We use HYCU for backups and while I was really skeptical about it in the beginning, it is absolutely solid in a Nutanix environment. Overall we are happy with Nutanix.

We have migrated from VMware to Nutanix, well, over half way there from a 8,000 VM pool. Nutanix has been a royal pain. LCM updates fubar'ing, Firmware upgrades screwing up UUID numbers of disks so CVM's won't boot, VM's not handling vlan nic and vxlan nic together, our SAN's just now being supported, the amount of bug fixes is weekly and impossible to keep up with, a team's ddos'ing of us with their app caused distributed storage to reboot every VM in the cluster, etc etc etc......I hate Broadcom but Nutanix has been painful!

OpenShift as an alternative to Tanzu.

OpenShift Virtualisation or whatever it’s called for the virtualisation part of VMWare.

Used to do those migration in a previous life.

The latter is IIRC rebadged KubeVirt

OpenShift Virt is way more than Kubevirt.

It is, I was thinking more of the core of it. In the same way one might say modern Openshift is rebadged kubernetes, but I will admit it wasn't best word choice

Running machines under basic Linux KVM (Kernel Virtual Machines). Veeam have started supporting it, but only on RedHat.

Reply on news.ycombinator.com