Hacker News Reader

Comment by betaby | Hacker News Reader

Are there plans to move k8s to nft? Seems like more flexible back-end compared to iptables.

This depends on the CNI plugin you're using. In particular, Cilium implements the entire firewall in eBPF.

Could you elaborate the benefit of doing it that way?

Disclaimer: I'm not an expert in Cilium! Only recently got into reading about it because of Jessie Frazelle's tweet about it. Still planning on deploying it into my small homelab.

It's my understanding that Cilium chose to do it this way because it allows low-level control of each network namespace that containers launch in, in addition to a high-level view of the system from the k8s API. This allows Cilium to build firewalling features that operate at a different level -- iptables/nftables filters on IP addresses and ports, but Cilium can filter on k8s resources and L7 protocols.

CNI plugins is one thing, while k8s subsystems itself still use iptables for KUBE-FIREWALL, KUBE-FORWARD, KUBE-NODE-PORT. At least that how I understand the k8s internals.

cillium reimplements kube-proxy's functionality. There may still be a couple of features missing, but it can definitely take over service routing more efficiently. I haven't checked in a bit but I'm pretty sure it covers NodePort too

nft and iptables are both packet filters. What is your use case?

It's a more operational question. In a year or two I plan to have all newly deployed servers with nft instead of iptables. So I would naturally expect to use the same nft commands to troubleshoot 'classic' servers and k8s nodes what sit on the same 'classic' servers - we run k8s on bare hardware.

Kubernetes implements load balancing via iptables probability rules. It creates "magic" static IPs on all kube nodes for ClusterIP services and makes iptables randomly select backend pods, making for a highly efficient zero-userspace load balancer. That gets less efficient with more rules due to the linearity of iptables. It also has an alternative implementation using ipvs, which is theoretically more efficient but has faced a number of issues over the years.

Kubernetes also has network security policy features for firewalling, that piece is implemented by plugins which do whatever they want though.

At this time, kubernetes requires you to be using iptables instead of nftables for all of this, so it will be interesting to see it running on iptables-free servers in the future.

Reply on news.ycombinator.com