FreeBSD kernel-mode WireGuard moves forward out-of-tree | Hacker News Reader

FreeBSD kernel-mode WireGuard moves forward out-of-tree

arstechnica.com

Ars Technica had a more in-depth follow up, "FreeBSD kernel-mode WireGuard moves forward out-of-tree" [0], and essentially it's just moving into Donenfield's own repository for maturation/testing until fully baked. It was announced on zx2c4 yesterday, ("WireGuard for FreeBSD snapshot 0.0.20210317 is available" [1]).

It appears that for some reason Macy, whom Netgate hired, spent a year trying to port the Linux kernel version (with ample kludging and ifdefs to make it work) rather than the more portable original core standalone version. The result wasn't great. But the rushed replacement Jason volunteered a lot of time for was, well, rushed, and everyone agreed that while kernel-mode wg in FreeBSD is very desirable the whole point of the project is to be really reliable and secure so worth taking more time to do right.

This presumably won't represent that much of a delay in the end. And while it's too bad Netgate couldn't have been more collaborative and gotten it right from the start, it's also impressive and humbling to see skilled people rallying to get it together in the end. Wireguard is such a great project.

----

0: https://arstechnica.com/gadgets/2021/03/freebsd-kernel-mode-...

1: https://lists.zx2c4.com/pipermail/wireguard/2021-March/00651...

I've been trying to stay quiet here because I just want to part ways, but the last sentences here got to me.

This wasn't a "mission to fix if_wg," at least it didn't start that way, and I think that it's important to acknowledge that my motivations here weren't exactly what's happened.

Let me start with: folks can hate me for this, but I generally like mmacy. I don't like specific things he did with this driver, but we usually get along. I don't really like how he's getting dragged around like this.

Now, this didn't start as a campaign to fix what was put into the tree, or to get Matt hit with the crap-bombs he's been dealt. This is the rough sequence of events:

- I use openvpn

- I don't like my openvpn setup, let's try if_wg

- oh, there are a couple problems here, let's fix those

- ok, what would it take to get wireguard-tools support?

This is where I get in touch with Jason.

- "oh, we gotta fix this"

- yeah, I can believe that

Cue hackathon session, we fix:

- All the jail bugs I can spot

- The race conditions we spot

- Number of panics

- Buffer overflow

- Privilege check

- Resync a lot of stuff with OpenBSD

- A number of things that I don't see, but I'm not a qualified expert in the area

Then we're here, where the stories start dropping and all hell breaks loose. For me, this isn't a story about how skilled people rallied to get it together, this is a story I'm not particularly proud of.

No further comment, because I'd like to get back to what I do. I know this isn't going to end well for me, so I likely won't check back.

I've run into the Netgate folks plenty of times, and so have many others. They are not nice people. Just look over Reddit or their Forums (especially during the AES situation where I was banned for life for a post saying I was unhappy with the decision even though I apologized).

I kept hand waving it away and using pfSense but no more. They do not deserve anybody's business. They are not good partners for the community.

My point, for you, is you did nothing wrong. The blame lays on their side.

What will you go with instead of pfsense?

I don't really care about this wireguard debacle (besides better code = good, and openvpn alternative = good), but I don't like the pfsense plus pro definitive paid game of the year edition.

I moved over to opnsense this weekend in a few locations. Will do more over time. In the few locations where an official appliance is used, I will likely move to swap those down the line.

Ive used pfSense since well before netgate even existed, and enough its not just in use in my home or lab. I generally dont made decisions based on bad PR or internet drama. So i didn't really bother to move over the AESNI stuff, or even the gnid/build tools etc. Though the gnid thing was what opened my eyes to what netgate was doing.

But their choice to diverge their code to basically closed source [1] and only contribute minimally to the CE, and leave it on people using CE to "enable" their features/changes leaves me with little choice but to move on. I use products like these because they are open to audits and fixes both for bugs and vulnerabilities. In the cases where I have used close source devices, especially at an edge location, its been with a trusted company with a storied history of security focus (like Cisco, Proofpoint, Palo Alto etc).

Netgates decisions on 2.6/pfsense+ basically mean that I would need to trust the security of the device to a small number of people that have a history of reacting very poorly to any question or criticism. And the pattern of moving their code base to something that isn't open to audit's/researchers eyes gives me practical reason to stop using or recommended their products. Which is something I find unfortunate. Its not just the wireguard thing in a vaccuum, its the pattern over time coupled with the choices they have made.

All that said my initial moves to opnsense have been mostly positive.

[1] https://www.netgate.com/blog/announcing-pfsense-plus.html

OPNsense, simple as that. It forked off pfSense back in 2015ish. It's not a perfectly drop-in replacement but it's close, has nice devs and community, and easily exceeds pfSense in certain regards (not least wg itself, topic related, since unlike netgate they don't have an issue with using one of the perfectly decent user space versions while waiting on a kernel version).

Like xoa says below, I'm probably going opnsense. There are other options (untangled, straight OpenBSD, VyOS), but for the switch I'd make I think it would be the easiest.

This seems to come up again and again. Too bad really. Seems like Opnsense is a solid alternative for those who want to avoid pfSense.

I didn't mean to spin this for Netgate. But Donenfeld hasn't seemed to want to make a big thing over it, and I just wanted to try to respect that for a summary. Kind of figured anyone really interested would read the stories and comments and get the gory details, but I'm genuinely sorry if you felt I disrespected the situation by being too breezy. And I do think there was an impressive rally to try to meet an admittedly artificial deadline and get something better in place rather then letting it slide or throwing more bombs then necessary. I do recognize it was serious.

I was linked back to your reply because it seems that I've not really communicated this very well- there's nothing wrong specifically with how you represented the situation, and I'm sorry if I came across as angry about you specifically. Read the rest of this in that same tone.

I'm angry that this has blown up like it has, and I realize that Netgate hasn't helped themselves out at all with the statements they've been releasing. If I was a PR person, my immediate reaction would have been "Hey, we're pulling this from the build. Know you guys were all excited about it, but points to press release"

I'm trapped here, you know? I can't speak for the proportions of how bad it was because I'm just a kernel guy, not a security guy. If I say "I don't think it was really that bad," I can pretty much immediately be written off as unqualified to make those kinds of statements.

We did end up nearly entirely rewriting the driver, but a significant chunk of that was removing iflib to fix a load of vnet issues and simplify it. I'm proud of what we ended up with, but I'm not proud of how this was handled by pretty much everyone around me.

Finally, to me, the deadline was very real. I thought we could end up with something that I'd be able to merge in time for 13.0rc3 (builds started today) in a relatively non-disruptive manner. It wasn't until most of the time was up that time had passed until I realized what we had come up with, and started hoping that I could still pull it off with significant testing.

It should tell you all you need to know about Netgate and pfSense when they hid the build tools without warning, changed the license, and hired a convicted residential terrorist like Macy who can't be trusted to show the restraint of civilized behavior. Use opnsense instead.

Ok, perhaps we'll change the article from https://www.netgate.com/blog/wireguard-removed-from-pfsense-... to that (thanks!). If there's a better article, we can change it again.

The original netgate blog post is about a slightly different event from arstechnica's follow-up (in this FreeBSD/WireGuard saga).

Arstechnica's article is about the removal of the 'rushed' WireGuard implementation that was supposed to go into FreeBSD 13 and the fact that development is continuing out of tree.

Netgate/pfSense had backported the old (pre-rushed) implementation that they had commissioned to the FreeBSD 12 kernel version which was released as part of pfSense 2.5 several weeks ago. They have now chosen to rip it back out again.

WireGuard is being removed from FreeBSD 13 (the base for pfSense).

There were some tit-for-tat messages between devs here, and while the implementation was probably fine, it is pretty reasonable to take a step back here, take stock, and take it from there.

https://lists.zx2c4.com/pipermail/wireguard/2021-March/00650...

pfSense is currently based on FreeBSD 12, not 13. And it only just moved to 12 a month ago.

Recent discussion about its addition: https://news.ycombinator.com/item?id=26475519

I'd read that WireGuard was supposed to be easier to implement because it supports fewer distinct crypto protocols and has fewer features overall compared to OpenVPN. But now this code is being pulled. Was it just a rushed implementation? Or is WireGuard harder to implement than initially thought?

There was concern about code quality, and accusations of what amounted to amateur mistakes in Netgate's particular implementation. I don't know how accurate those accusations are, though.

They were made by Jason Donenfeld, the author of WireGuard which is of exemplary quality so I take it pretty seriously :)

It's possible to browse the before-changes-started version of the FreeBSD code, through either CVS or the FreeBSD Git mirror. To save people the effort of finding the right git revision and the path, the kernel module starts here: https://github.com/freebsd/freebsd-src/tree/95331c228a39b44c...

On a casual inspection, there are at least kernel printfs in crypto code in __chacha20poly1305_decrypt (in module/crypto/zinc/chacha20poly1305.c) that were not in the original version of this from Linux.

The original version would be GPL v2 right? If that's the case it'd make sense that the two don't match because you can't reuse the code for FreeBSD. You'd want a completely clean implemention just to avoid any appearance of impropriety, unless the new implementation was done by the copyright holder themselves.

That particular file seems to have been taken from Jason A. Donenfeld's original, which was dual-licensed GPLv2 or MIT and so legal to import (under MIT) into FreeBSD. I don't know which upstream version of the file it comes from, but it's definitely very close to the version in https://github.com/WireGuard/wireguard-monolithic-historical .

Wireguard works really well in OpenBSD since like November. I find it WAY less painful to use than openvpn. I switched clients over to this implementation and they're happy as clams. Incredibly easy to set up and use, all out of a base install of a rock solid operating system that got this introduced without any kerfuffle. I think one of the weirdest sticking points about how ridiculous this whole situation is how they didn't even try to write this with that already-done implementation in mind, even after this was suggested by jason... It would have made their job a lot easier

> I think one of the weirdest sticking points about how ridiculous this whole situation is how they didn't even try to write this with that already-done implementation in mind, even after this was suggested by jason... It would have made their job a lot easier

Absolutely Agree.

The whole thing is pretty crazy and makes me lose even more respect for Netgate than I had already[1].

Also, as a long time FreeBSD user, it also makes me worried about the quality of other things in FreeBSD now, if this was _almost_ allowed to get shoehorned in, in such bad condition. Not sure if this is just an unlucky one-off, or a sign of deeper problems in FreeBSD development (not enough devs, process problems, etc).

[1]: the whole opnsense stuff was super distasteful https://en.wikipedia.org/wiki/OPNsense#cite_note-7

Someone from netgate implemented it poorly and then the original authors tried to fix it in a hurry but ran out of time. There was also some mailing list drama I think.

See https://news.ycombinator.com/item?id=26475519

Just checked and there isn't any pfSense update available, so even if they removed it from the source code, given that majority doesn't compile pfSense themselves, I'd argue that it isn't removed just yet.

Is there some actual concern about WireGuard itself or just the FreeBSD implementation?

Apparently the original implementation had issues. Apparently NetGate shipped some bad code after hiring a FreeBSD contributor for their implementation. Kyle Evans has even decided to step away from maintaining wireguard-freebsd. Seems like a mess.

> Kyle Evans has even decided to step away from maintaining wireguard-freebsd

That is not true. Everyone acknowledges the code is not up to the required standard so it is being removed but Kyle and others will continue to work on it with the aim of adding it back.

Edit: as @asmr below linked to, he has has since announced that he is quitting.

https://lists.zx2c4.com/pipermail/wireguard/2021-March/00652...

That's a real big shame. I know he doesn't blame Netgate, but I will. If it wasn't for their poor attitude (par for the course from Netgate), this would not have blown up.

I think it's just lack of formal peer review [1]

[1] https://lists.zx2c4.com/pipermail/wireguard/2021-March/00650...

To state this clearly: there are not concerns about WireGuard itself.

To be double-clear - the problems being discussed are in the freebsd kernel-mode implementation. The freebsd userspace implementation is fine, and implementations on other OSes are fine :)

There were questions around the implementation. See https://news.ycombinator.com/item?id=26475519

I don't get where all the hype about WireGuard comes from. It's just another VPN, of which we already have several perfectly well working ones. What is so special about it over all the other things that go into Linux and FreeBSD, and that actually do something new and unique and don't make the news?

I'm personally ready to ignore it and add it to my BS bingo card. Blockchain anyone?

I would suggest watching https://www.youtube.com/watch?v=88GyLoZbDNw .

As someone who just wants to maintain a few VPN tunnels without having to be an expert in OVPN/IPSec/whatever, the ease-of-setup and conceptual simplicity of WireGuard is like a cup of ice water in hell.

The order-of-magnitude better performance and excellent security properties are just the icing on the cake, as far as I'm concerned.

For those who care, the podcast 2.5 Admins had a nice discussion about the WireGuard 'drama'.

https://2.5admins.com/2-5-admins-30/

Reply on news.ycombinator.com