For AWS and GCP, there's zone-specific VM scaling groups that ensure you're in multiple AZs so you'd know for sure you're not on a single physical server.
"You can easily get a box that has this many (or more) cores. I wouldn't be surprised if our cloud provider analyzed the topology of the connections, and decided to put the whole thing on the same server."
And the comment you replied to is referring to "the server"
The question was how you reboot the posited singular hardware server with no downtime to any VMs running on it.
It involves copying the whole VM image over, and rewiring the network connections virtually on the fly.
I say (nearly) because it's not 100% transparent, but my understanding is that it works properly the vast majority of time.
If the primary catches fire or crashes, the secondary boots the VM quickly without data loss.
If the primary reboots, it checkpoints RAM to the secondary pauses the VM and unpauses it on the secondary a few milliseconds later.
Note that this transparently handles storage replication, which is something that is notoriously difficult in Kubernetes.
If your cluster fits on one machine, you've paid for a lot of (currently) unnecessary complexity up front, both in dev time and in hardware cost.
If your app scales to need the cluster, congrats. Sometimes delaying time to market to allow for a smoother ramp makes sense. Sometimes it does not.
(Wikipedia is a good example of succeeding without ever needing to scale out the back end. I doubt they'd have won out over their competition if they took an extra 12-24 months to launch.)
So you don't have one VM's, but two. In case of Op's scenario you are paying for 196 CPUS instead of 98.
> If the primary reboots, it checkpoints RAM to the secondary pauses the VM and unpauses it on the secondary a few milliseconds later.
So a very small downtime, but not 0.
> You can easily get a box that has this many (or more) cores. I wouldn't be surprised if our cloud provider analyzed the topology of the connections, and decided to put the whole thing on the same server.
I wouldn't want to have paid for the engineering that went into that if you could actually do that.
Yes. GCP literally has a button that allows you to Replace/Restart all the instances in an Instance Group. Our deployment system just pushes a new GCP template and GCP does all the work rolling it out. No Kubernetes.
However, a permanent machine failure or unexpected reboot implies a few seconds or minutes of downtime.
These systems usually use a local disk, a synchronously updated disk across town (low latency, but on a different power grid, outside most natural disaster blast radii), and a far away asynchronously updated disaster recovery disk.
The disaster recovery disk provides a crash-consistent state that's less than a few seconds out of date.
These days, the "disks" tend to be deduped and compressed SSD's that use parity encoded raid. Their hardware cost is lower than a single copy on ext4, but they come with an enterprise tax / support contract, etc.
In practice, it's durable unless corporate HQ is wiped out. If that happens, it won't be the weak link in your business continuity plan (but a few customers' orders might've been dropped).
This stuff is tremendously unsexy, but it'll plug along just fine until you have some workload that can't be partitioned, and that needs more than roughly 40-100gbit of network/disk bandwidth, 128 cores, or 1TB of DRAM.
(Edit: Forgot to mention that the the apps run inside a VM infrastructure that supports live migration.)