Show HN: Netfence – Like Envoy for eBPF Filters | Hacker News Reader

Show HN: Netfence – Like Envoy for eBPF Filters

github.com

58 pointsdangoodmanUT7 comments

To power the firewalling for our agents so that they couldn't contact arbitrary services, I build netfence. It's like Envoy but for eBPF filters.

It allows you to define different DNS-based rules that are resolved in a local daemon to IPs, then pushed to the eBPF filter to allow traffic. By doing it this way, we can still allow DNS-defined rules, but prevent contacting random IPs.

There's also no network performance penalty, since it's just DNS lookups and eBPF filters referencing memory.

It also means you don't have to tamper with the base image, which the agent could potentially manipulate to remove rules (unless you prevent root maybe).

It automatically manages the lifecycle of eBPF filters on cgroups and interfaces, so it works well for both containers and micro VMs (like Firecracker).

You implement a control plane, just like Envoy xDS, which you can manage the rules of each cgroup/interface. You can even manage DNS through the control plane to dynamically resolve records (which is helpful as a normal DNS server doesn't know which interface/cgroup a request might be coming from).

We specifically use this to allow our agents to only contact S3, pip, apt, and npm.

Open articleView on HN

The first sentence of the README is:

  Like Envoy xDS, but for eBPF filters.

Which would make the title make much more sense!

I agree.

I thought about putting xDS in, but I worried it might be confusing for people who might not know the xDS specifics of Envoy. But now I'm second guessing it lol.

Cool! While in Kubernetes you have cilium that does basically the same thing, outside of Kubernetes I've been using explicit proxies to do this kind of thing, which requires applications to support http proxy. I could definitely see transitioning those workloads to using ebpf filters instead.

Any fundamental reason you can't allow/block individual ports, or just a design choice?

Neat. One issue I’ve encountered with lookup-based rules is the latency of updating the client’s name caches when records become stale. How do you handle that here, or does it need to be done in L7?

For looking up the IP or whether you are permitted for some host?

For the former you don't, it's just DNS. The local DNS server respects TTL, and is no more expensive than a normal DNS lookup. It just proxies it to take the resolved IPs and push them into the eBPF map.

For the latter, the default expectation is that you push the rules to the "Attachment", typically in the "SyncAck". If you need to make updates, you push down deltas (add/remove rule).

You _can_ do dynamic DNS resolution, and there you'll be paying either 1x or ~2x DNS depending on whether your control plane already knows the IPs.

If you are running kubernetes, is there any reason to use this over cilium? What you are doing sounds very similar to what cilium does.

Maybe not, but we're not using k8s for our agent VMs

Reply on news.ycombinator.com