Anything breaking security of your OS kernel or some trusted process will still be able to steal all your data, since it's executing on the CPU with access to the keys.
Any attacker with physical access can still grab a hold of some PCIe or LPC port somewhere and try to convince your IOMMU to let it DMA out all memory. Or just manipulate the BIOS to install a permanent rootkit.
Why is this not using per-page or per-process keys, in some kind of secure storage? That'd actually add another barrier even if the kernel is already compromised. And the CPU does actually support this... but only for VMs with SEV, it seems. Would be nice to extend this.