I'm pretty impressed with the GCP response, both the fact that they identified the behavior and took the first step in reaching out.
With seccompbpf it's pretty simple to have systemwide tripwires on certain files/syscalls/network operations. Even if the attacker gains root, your tripwire will probably alert you before they can disable it.
Likewise, GCP Dataflow quite trivially allows you to escape onto the worker machines and take the (huge) binaries that implement it. They have some really nice detailed status pages!
In dataflow's case, container is not treated as the boundary. And there are several important things to note:
- Dataflow's VMs are in customer projects, so there's no risk of cross-tenant access.
- When launching dataflow jobs, the launcher identity is checked to have iam.serviceAccountUser IAM role, which means that the identity should be able to launch a VM with the same service account just fine. So dataflow is not escalating the permission beyond GCE VMs.
- Just as VM launched by someone, if anyone else can log onto those VMs are controlled separately.
- Container is used in dataflow only for convenient image delivery, not for a security barrier. VM is.
There's two types of cloud providers, the ones who take security seriously and the ones who learn security the hard, public way.
I'm a bit surprised that Azure would get lumped in with the other cut-rate providers but that's becoming more and more obvious with the vulnerabilities of the past few years.
The vms are somewhat hidden in the UI iirc but otherwise you can enumerate them via API and ssh to them and debug/profile (which I was doing to get cross-language profiling on data flow pipelines with py-spy and jvm perf output).
It's just a worker vm in your project.