Hacker News Reader

Hacker News

Comment by TurningCanadian | Hacker News Reader

Parent Full thread

TurningCanadian

Check out

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-...

if you're running nginx. Consider setting it to true instead of the default (false)

---

By default the NGINX ingress controller uses a list of all endpoints (Pod IP/port) in the NGINX upstream configuration.

The nginx.ingress.kubernetes.io/service-upstream annotation disables that behavior and instead uses a single upstream in NGINX, the service's Cluster IP and port.

This can be desirable for things like zero-downtime deployments .

View on HN

You just moved problem from ingress controller to kube-proxy (or it's replacement) on the same node. Race condition is still present as before.

This is not a good idea, you're doubling the load on conntrack.

Maybe ingress-nginx should fix their config generation instead. Had a cluster with at least one change per second to the ingresses, and nginx would regularly just die (and orphan existing requests after 20 seconds).

ah good tip, I don't care about session affinity or custom balancing algos so that works. I'd imagine running in GKE or AWS you would also avoid the DNAT / conntrack overhead as pods by default use a routable VPC IP instead of a magic CNI IP. Would have to test that though.

Quote from related issue:

The NGINX ingress controller does not uses Services to route traffic to the pods. Instead it uses the Endpoints API in order to bypass kube-proxy to allow NGINX features like session affinity and custom load balancing algorithms. It also removes some overhead, such as conntrack entries for iptables DNAT.

How does this remove conntrack overhead? It should add more, because the node with the ingress controller now has to hold an extra <Cluster IP Cluster Port, Pod IP, Pod Port> mapping, regardless of what CNI is used (flannel in gateway mode also eliminates this overhead, by the way - you just need to make sure there is nothing like the default EC2 source destination check in place).

Reply on news.ycombinator.com