Hacker News Reader

Comment by jaxbot | Hacker News Reader

I'm waiting for GCP to support preemptible GPU instances. Would love to be able to spend ~50% less on GPU instances when running a batch job that can be stopped and reloaded.

View on HN

Not sure that will happen anytime soon. It's the same with Local SSD. The virtualization of this environment is very challenging, maybe beyond a point where it makes sense for them to do so.

But GCE already has local SSD’s for preempt instances, and they even cost less.

Wasn't aware they added the support. Well, maybe they can figure out GPUs too.

I'd be surprised if it'd be that much additional work, if the local SSD options are NVMe -- which is also PCI-e like GPU's.

Wait what, would they expose the raw SSD pci device to your vm? What's stopping you from scraping all the leftover data from the previous customer?

Probably something along the lines of secure erase. Most modern SSDs/NVMe drives are encrypted by default in firmware. All the firmware needs to do is throw away the old keys and generate new ones. It's better than zeroing the drive as there is no wear to the write cycles and guarantees that the slack space in the SSD is also cleared, which DD'ing to /dev/nvme0 wont be certain of. The nvme-format tool can be used for this: http://manpages.ubuntu.com/manpages/zesty/man1/nvme-format.1...

On newer SSDs, the sanitize command would be preferable for this use over the format command. IIRC, the format command doesn't require quite as strong a security guarantee as the sanitize command: the latter ensures that user data is cleaned from both the flash and all buffers, CMBs, etc.

I've seen some hints that GCE uses NVMe emulation not PCI passthrough. This would allow the hypervisor to implement features like live migration.

I find it interesting to think about virtualization on a spectrum with passthrough at one end and pure emulation at the other. Particularly when applied to I/O peripherals, if the bulk of the I/O can effectively be classed as payload, stepping slightly away from passthrough while maintaining roughly the same layer of abstraction grants the implementer considerable liberties in implementation with very little overhead.

If one were to look at storage, specifically, the move to 4k block sizes would seem like a particular boon in terms of increasing the volume of data covered by any given IOP.

Disclosure: Jon works on virtualization for Compute Engine :). (and I work adjacent-ish)

Your data is encrypted as soon as it exits the VM and before it's written to storage.

https://cloud.google.com/compute/docs/disks/#ssd_encryption

Reply on news.ycombinator.com