Yes, that's an option -- the simplest option would be a single overlay network, with the node-node tunnels encrypted using IPSec or similar (https://github.com/coreos/flannel/issues/6 or https://www.weave.works/documentation/net-latest-how-it-work... or https://github.com/projectcalico/felix/issues/997). I think this would be tricky to configure in GKE though.

With a secret-per-pod, your key material lives in in the etcd on the API server, and gets mounted in a tmpfs on each pod that is given the secret. Only Pods in the Secret's namespace can access the Secret, so if you have RBAC configured correctly it should be possible to lock this down tightly to only the code that needs the Secret. (I'm not sure how to do this in GKE; I'm currently treating each cluster as a single security domain).