Hacker News Reader

Comment by bogomipz | Hacker News Reader

Indeed and if you bring up a Kubernetes cluster today on AWS using their EKS there is no overlay network. There's a CNI but no overlay.

View on HN

In some deployments we remove the VPC CNI and use an overlay anyway because of the integrations we need. You do lose Security Groups but that's not a big deal if you aren't using them in your deployment anyway.

Oh sure there's lots of good reasons for not using their CNI not least of which is needing to worry IP space utilization. I was just trying to point out overlays aren't a requirement.

I would need to double check, but isn’t it still running an overlay? Just one which is transparent to you?

Well an overlay network by definition is always going to be transparent to you. But no the AWS VPC CNI does not create an overlay. It's just layer 3. It works by adding ENIs to your worker nodes with secondary IP addresses on them. And those secondary IPs are from your VPC's address space. See:

https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/c...

Yep, another reason it does that is to spread the risk and limit the need for tons of unchecked (no src/dst check) addresses in your VPC.

In the end there is no ideal scenario; boils down to what works best for the use case (or what is the 'least worst' solution). Sometimes it gets you down, but those imperfections can turn a churn job into an interesting one.

Reply on news.ycombinator.com