Hacker News Reader

Comment by remus | Hacker News Reader

> The offerings between the two cloud vendors are also not the same, which might relate to their differing response times. GCP allows you to attach a GPU to an arbitrary VM as a hardware accelerator - you can separately configure quantity of the CPUs as needed. AWS only provisions defined VMs that have GPUs attached - the g4dn.x series of hardware here. Each of these instances are fixed in their CPU allocation, so if you want one particular varietal of GPU you are stuck with the associated CPU configuration.

At a surface level, the above (from the article) seems like a pretty straightforward explanation? GCP gives you more flexibility in configuring GPU instances at the trade off of increased startup time variability.

View on HN

I wouldn't be surprised if GCP has GPUs scattered throughout the datacenter. If you happen to want to attach one, it has to find one for you to use - potentially live migrating your instance or someone else's so that it can connect them. It'd explain the massive variability between launch times.

Yeah that was my thought too when I first read the blurb.

It’s neat…but like a lot of things in large scale operations, the devil is in the details. GPU-CPU communications is a low latency high bandwidth operation. Not something you can trivially do over standard TCP. GCP offering something like that without the ability to flawlessly migrate the VM or procure enough “local” GPUs means it’s just vaporware.

As a side note, I’m surprised the author didn’t note the amount of ICE’s (insufficient capacity errors) AWS throws whenever you spin up a G type instance. AWS is notorious for offering very few G’s and P’s is certain AZs and regions.

Fungible is selling a GPU decoupling solution via PCIe encapsulated over Ethernet today, so it can certainly be done.

And NVIDIA's vGPU solutions do support live migration of GPUs to another host (in which case the vGPU gets moved too, to a GPU on that target).

I doubt it would be setup like that. Compute is usually deployed as part of a large set of servers. The reason for that is different compute workloads require different uplink capacity.You don't need a petabyte of uplink capacity for many GPU loads but you may for compute. Just switching ASICs are much more expensive for 400G+ than 100G. That hasn't even got into the optics, NICs and other things. You don't mix and match compute across the same place in the data center traditionally.

I've only ever used AWS for this stuff. When the author said that you could just "add a GPU" to an existing instance, my first reaction was "wow, that sounds like it would be really complicated behind the scenes."

Reply on news.ycombinator.com