Hacker News Reader

Comment by ehonda | Hacker News Reader

Say you created 32 dual vcore VMs with this, each with 2GB of RAM. How would they perform simultaneously together, vs a separate dual core PC (like a core2duo)? I don't know very much about the performance differences or if its even possible to compare such a thing.

View on HN

Probably not near as well, if they're all actually doing anything. You'd hit bottlenecks moving data around, the bus used to fetch memory isn't 32 times as wide, and the memory isn't 32 times as fast, same with other resources.

You could in practice run quite a few VMs before you'd run into trouble, especially if most of them are idle at any given point.

It's still much faster than any Core 2. It's faster than 2 cores of Sandy Bridge on my older 2950x, so I'd expect 39z0x to be even faster, probably beating older Skylake.

Yeah, I'm sure it's going to depend a lot on workload. If each one is in some crazy tight loop, CPU bound without using much working set of memory at all, it should be faster.

Or more realistically if they're usual VMs and idle the vast majority of the time, that should be faster too.

That's some data I'd love to see. 32 1u servers all connected with fiber or 10gig vs 32 VMs on the same host vs 32 k8s pods on the same node. I'm wondering if the same test would scale from 4 hosts/VMs/pods

I don't even think there's a benchmark out there for such a test.

VMs in general are more memory-bound than CPU-bound (exceptions for things like SQL servers, encoders, etc). Hypervisors are generally pretty good about spreading VMs across a pool of CPUs and grabbing whichever is idle at the time. You can manually set affinities to always use specific cores, but it's generally wasteful to do so.

One caveat (at least with how vSphere 5.x worked) is the hypervisor has to claim all CPUs at the same time in order to do work, even if the other guest CPUs are idle. For example, if I have a 4 core VM on a 6 core host, it has to wait for 4 of the 6 to be free before the VM gets to do anything. So sometimes VMs with less CPUs can outperform one with more for the same workload. Getting proper measurements on your loads (peak/avg CPU, memory, disk IOPS etc) is critical to a good migration.

You might want to to check out this Linus Tech Tips video from December:

https://www.youtube.com/watch?v=jvzeZCZluJ0

What he did was he took a single 32 core AMD CPU to replace all the computers in his house, including gaming PCs. At around 00:54, he mentions that the cores on the CPU are not "weak cores".

Didn't they also build like a 6 person gaming rig on one server?

The VMs, in addition to dealing with the normal VM overhead we all already understand, would have much lower (like an order of magnitude) bandwidth to main memory and a similarly smaller share of L3 cache, in both cases because the resources are shared across all 32 devices.

So the answer to the question depends heavily on how cache-resident the problem you are throwing at them is. They'd do great mining bitcoin and be a total disaster as memcached hosts. More typical workloads will be somewhere in the middle.

Depends on target dual core machine spec. If all VMs run in full or heavy tasks you might face issues with scheduler. But I believe with lot of tunning on host like using numa, cpu passthrough, pinning and using light guests you might get close to those numbers with reasonable workload

Would you need 32 V's or could you run the program on one server with 32 threads or instances?

Reply on news.ycombinator.com