Hacker News Reader

Comment by nick__m | Hacker News Reader

Before my vacation we (3 colleagues and myself) completedan 8 months long migration (coordination with stakeholders is longer and more complex than migrating a 192TB VM !!!) to 6 proxmox clusters so 20 to 40 clusters for 40k is certainly possible but imo it would be unwieldy.

View on HN

> 192TB VM

Why? Honest question, what leads to that Kind of size and why cant it use NAS shares or SAN disks for most of that data? Kudos on the migration!

I wish I knew ... at least it was a span volume so we could use proxmox support for vmdk and achieve about 30 minutes of planned downtime but a week of storage vmotion followed by another week of the proxmox equivalent.

When I get back to work I will finish the samba configuration of the ceph cluster front-ends to replace those elephants.

Would you do it again (ProxMox)?

The FC multipathing was a learning experience and the manual workload placement requires good metrics on your workloads. The built-in ceph is decently configured and more performant than FC if you have 100GBS mellanox NICs and adequate quantity of ram. Veeam integration is serviceable but it's not as mature and polished as the integration available on vmware.

Having tried azure local before (it seems magical but the more you use it the worst it gets, update failed for no apparent reason on only some supposedlyidentical nodes, the sdn was atrocious to deploy and was manageable from wac only), I would recommend proxmox over it anyday.

If you don't have linux expertise on hand and have traditional FC based storage, I would recommend something else, probably nutanix if your budget is big enough.

And if you had to do it one a normal human setup of three hosts and 15 total VMs, no fiber, no san, no special nics or anything like that, just tied in to an ordinary Veeam with the ssd storage on an immutable Linux machine elsewhere?

I would choose promox or ovirt (oracle virtualiztion manager is pretty nice, I use it because the licence of the only oracle database of our institution requires it, that and a specialc cores pinning configuration, to avoid licensing every cores in all clusters, https://houseofbrick.com/wp-content/uploads/2015/03/Oracle-P... ) but it's oracle so we went with proxmox.

Reply on news.ycombinator.com