Virtualize Your Network on FreeBSD with VNET | Hacker News Reader

Virtualize Your Network on FreeBSD with VNET

klarasystems.com

85 pointsvermaden21 comments

I'm always exasperated how an iPhone or any other smartphone cannot properly check if a WiFi has functional internet connection before switching over from cellular data. Half a century of relying on networking abstractions — and we're basically unable to say, hey, let this particular process work on this network connection while the rest of the system is relying on the other adapter. VIMAGE gracefully and painlessly allows that.

That seems semitrivial on Android since at least a decade back, you just need to implement it and find all the edge cases, so I guess there is a reason this is not done already.

e.g . one routing table per interface

add rules in global routing table for each new interface that is started:

  ip rule add from/to $IP_OF_IFACE/32\ 
    table $UNIQUE_RT_FOR_IFACE
  ip route add default via IP_OF_IFACE_GW/32\ 
    dev $IFACE_NAME table $UNIQUE_RT_FOR_IFACE

Then when something changes on the iface that is not defaul; 1. remove old configs for that iface, 2. open a socket on the new ip on that iface to test internet connection. 3. If ok: change default in main routing table to the GW that iface uses.

On sane OSes you could manually modify the routing tables. On iOS you can't even do basic things like turn of discoverable mode for bluetooth so this shouldn't be surprising.

FWIW, this is how my imds-filterd (https://github.com/cperciva/imds-filterd) works: Traffic for 169.254.169.254 is routed into a VNET jail. That way I can have a web server which proxies IMDS requests while staying out of the way of any networking being done on the host system (e.g. anything else which might want to bind to port 80).

I’ve dabbled with OpenBSD and FreeBSD for personal projects for a long time, but have never had the opportunity to use them for anything “serious”. I am trying hard to figure out a path to change this.

ZFS, Jails, VNET, and the stellar security track record are very enticing. In-kernel wireguard support was the last missing piece for me.

The elegance and simplicity of the BSDs are becoming increasingly apparent in contrast to the Linux world.

As an example, compare the jails UX to say, LXC/LXD…

I used to chase bleeding edge Linux distros but these days, I nearly always reach for either an LTS release or a BSD for my personal projects.

Sadly, I got all my network appliances setup and configured with Ubuntu 20.04 LTS a month or so before OpenBSD merged their WireGuard kernel module. I'm too lazy to touch those servers given they're relatively low traffic, especially during Pandemic times as I'm always on my local network.

I need to try OpenBSD again, I always got an issue where the screen would hang after X amount of time and couldn't figure it out.

On the off-chance you were using amdgpu, they merged a huge driver update in 6.8 that improved reliability for me from "unusable" to "easy daily driver."

Why is in-kernel Wireguard a priority? I'm happily using https://github.com/WireGuard/wireguard-go with no issues to speak of, on FreeBSD.

In my experience using the userspace daemon on both BSD and macOS there was increased cpu overhead and slightly higher network latency compared to Linux with a module or built in.

Ah, perhaps. I haven't measured it; I only use FreeBSD on "applicance-ish devices," i.e. my router and NAS, both of which are pretty overspecced for CPU.

I got a chance to use FreeBSD in such 'serious' case:

https://vermaden.wordpress.com/2019/06/19/freebsd-enterprise...

... I recently added Jails there with Minio inside in each of those jails to serve S3 storage.

Works like a charm.

Very comprehensive write-up. Thank you.

Thanks.

It always seemed to me that FreeBSD in particular could have all the pieces needed to implement the container runtime interface (and thus be able to run containers) but no one ever bothered.

That's kind of a pity, because FreeBSD could be an ideal OS to run containers: a stable system with unified APIs, without 1.8e308 different distros, most of them subtly incompatible with each other.

This is close to one of my pet peeves about FreeBSD: They're happy to say that they've had containers for longer than Linux and that Jails are better than Linux containers, and I don't really disagree on technical merit, but they seem to completely overlook that UX and images are Docker's killer features and FreeBSD doesn't have an answer to that[0]. I personally think that implementing a jail-backed OCI runtime and getting docker running on top of it (all of which should be fairly simple to do) would be a huge boost to getting more users - bonus points if it can run FreeBSD-native and Linux images by using the existing Linux-compat layer. I've even looked into actually implementing this myself, but the runc-containerd-docker codebases were too much for me to make sense of so far, unfortunately.

[0] Actually, partial caveat: cbsd and bhyve are at least meant to fill this space, I think, but they're neither a perfect match nor as easy to use IMO.

Maybe the FreeBSD project doesn't care about getting more users? It's certainly not a big motivating factor for OpenBSD, but I don't know how FreeBSD feels about it.

I can't speak for anyone else, but from my perspective (and I suspect others are similar): I like the idea of a jail-backed OCI runtime, but it's not interesting enough to my personal use to overtake any of the other OS-adjacent projects I'd otherwise work on in my spare time.

I could be convinced (well, paid) to make it a higher priority, but ultimately it's currently at the bottom of a barrel of much more enticing fish since I'm not really in the demographic that that kind of stuff targets.

Quite:) And seriously, maybe one of these days I'll find the time to understand how docker works well enough to retrofit it onto FreeBSD, but, indeed... it takes significant time and/or money, and this is currently just a hobby-interest, so there it sits...

SmartOS might interest you. It's not BSD (derived from Solaris), but it is marvelous for OS-level containers.

Is it still being developed? Joyent doesn't seems to be getting any headlines after they have been acquired by Samsung.

Reply on news.ycombinator.com