WireGuarding the mainline | Hacker News Reader

WireGuarding the mainline

lwn.net

206 pointsthinxer95 comments

Quoting Linus Torvalds:[0]

> I see that Jason actually made the pull request to have wireguard included in the kernel.

> Can I just once again state my love for it and hope it gets merged soon? Maybe the code isn't perfect, but I've skimmed it, and compared to the horrors that are OpenVPN and IPSec, it's a work of art.

0) https://lwn.net/ml/linux-kernel/CA+55aFz5EWE9OTbzDoMfsY2ez04...

Edit: fixed URL

That's great to hear, but I wonder if anyone has insight on the rest of the article. Is this Zinc API a prerequisite for WireGuard to be included? Is the fact that Zinc won't support hardware encryption a no-go or is the expectation they give reasonable? Has the kernel had any large API refactoring similar the crypto/Zinc (I would assume it has)--how did they play out?

There seems to be broad agreement for the need for a set of software crypto implementations with a straightforward synchronous, static dispatching API. The discussions seem to be mainly around code organisation and plumbing that API as an underlying implementation used by the dynamic dispatch asynchronous crypto API.

Right. So basically v2 of the Zinc patchset will address the nice points brought up on the list, and I think things should proceed nicely.

Jason: Any word on the name yet? My concern is the confusion with the existing my minizinc and flatzinc project, a standard constraint modeling language, which has nothing to do with crypto. There should be some better name for the Linux crypto lib.

So what? Name clashes happen all the time. Before seeing this comment, I had never heard of minizinc and flatzinc. In fact, if you had asked me if Zinc clashes with anything, I'd have thought of Tinc (which is a VPN product, so roughly in the same space).

An incredible compliment to be sure.

But in true Linus style, he just has to take a dig at another team at the same time to balance things out.

To be fair, his comment was not directed at another team, but at another body of code. (Two, actually.) The context makes it not merely a dig, but constructive feedback: "Those past projects have significant flaws, and here we have an example of how to do it right." If I were an OpenVPN or IPSec developer, I think I'd swallow my pride and take this valuable opportunity to learn how to improve my work.

Or specifically as whole systems with multiple implementations. There's no "IPsec" code really.

Sure. But isn't it a valid dig?

I don't know. I'm sure the teams working on their tools had their reasons.

There are reasons to have technical debt. Having reasonable reasons don't make the technical debt any easier to fix.

I don't think Linus' comment was directed at the author, it was at the code. Whatever the reason, the code he took a dig at is messier than wireguard.

You mean, each individual member of those teams had their reasons?

Not sure. I'm not on the team, but I seriously doubt their intent was to write bad software. I know before ipsec existed there was nothing, so why not thank the individuals who worked on it for pioneering the effort in the first place?

I think it's okay for him to be frank about things. We all don't like when he bites someone's head off; fine, that's not what he's doing here.

We're really at a point where people would prefer to discuss his tone rather than the content of what he says, even here.

That's because he can't make a single comment on a thread without at least throwing a least a couple of backhanded insults in there.

Fixed link: https://lwn.net/ml/linux-kernel/CA+55aFz5EWE9OTbzDoMfsY2ez04...

Thanks :) I'm not sure how that happened.

I'm really looking forward to start experimenting with Wireguard. Jason A. Donenfeld is also the creator of my favorite password manager https://www.passwordstore.org/ and recently I found a neat web frontend for git called cgit, lo and behold, when I looked it up I saw that Jason was the creator. He makes some really cool high quality stuff.

> He makes some really cool high quality stuff.

Couldn't agree more, all of his software is really nicely designed and implementated. Moreover I had a chance of exchanging emails with him and he's a really nice person.

He really is. We just got done hanging out with him at Black Hat, and he's one of the nicer people I've met here. A+++ would hang out again.

A couple of days ago I started testing with WireGuard and installed it to my Omnia Turris router. Mullvad provides WireGuard servers[0] for testing with a reasonable price, and I've been routing all the traffic from our apartment through WireGuard without any problems. The speed is just amazing after seeing the disappointing performance of OpenVPN, I can easily push 300-400 MB/s through the router, finally removing the last reason to not use a VPN for all the traffic.

[0] https://mullvad.net/en/guides/category/wireguard/

> I can easily push 300-400 MB/s

Wow. (For Wireguard performance and the link speed.)

10 Gbps link or just on the local network?

I presume a typo....

AFAIK 10 Gbps is the highest consumer internet bandwidth in several countries. So it's still possible.

Does anyone know why reverse christmas tree order of declarations (i.e. longest to shortest line length) is part of the style guide?

I don't know in their case, but I know it's the style used by Hisham Muhammad (author of htop and LuaRocks among other things), and he explained why here: https://hisham.hm/2018/06/16/when-listing-repeated-things-ma...

I’m wondering that too. What happens when a variable is initialized as a consequence of the state of two other variables (which would tend to be a longer statement)?

That this has to be in the kernel is a failure of the Linux architecture. This is middleware.

The only reason for it to be in the kernel is performance. There are userspace implementations already, like on Windows and macOS.

Because everyone knows kernel code runs faster than userspace code, right? Er no, because of architectural problems.

If you're going to argue this then I agree with tptacek, you must also think it's an architectural problem that filesystems cannot all be implemented over FUSE with performance equivalent to their kernel versions.

Yes, TUN is not 0-copy. Is there an equivalent that's as fast as the kernel in any other major OS? (I'm asking that honestly, I have no idea.) If there is, is in-kernel networking on that OS as fast as on Linux?

The Linux network stack is what it is, there have been recent improvements for it (e.g. DPDK), but I think it must not be so bad given it powers most of the Internet.

It doesn't have to be "over FUSE". There are microkernels out there like QNX and L4 which solve the filesystem in userspace problem, are extremely fast, and have done this for decades. You likely have a realtime L4 instance in your phone doing the real work talking to the phone network. Heck even my Nintendo Switch has a simple microkernel OS, with an unspectacular ARM processor, and it runs games - one of the most performance sensitive applications that regular people use.

Right, but L4 isn't even an operating system; it's an OS fabric, on which OS personalities are built. Nobody doubts you can build a microkernel architecture where filesystems and VPNs are equivalently performant in userland and the kernel, but nobody has managed to do that for a mainstream OS.

It's telling that xnu started out on Mach with a FreeBSD "personality", and grew into another shaggy monolith.

The L4 in your baseband or your enclave is not running anything resembling a general purpose operating system.

So I guess my argument is: sure, you could design and build VPNOS, the OS where VPNs are fast and never need to touch the kernel. But nobody wants that OS.

Minix. Rumor is it runs on a lot of Intel CPUs...

What is the performance of Minix running general desktop and server workloads?

I won't argue against you here, I like microkernel desing à la QNX, especially with synchronous message passing (MsgSend), and I used to run Minix3 as my main OS some years ago. It relies on a completely different scheduler model though, with different trade-offs. And you can't always avoid copies.

Also, QNX has never seen usage as a general purpose OS on a PC, even though I remember trying a QNX Photon Live CD years ago. I wonder what would work well, and what wouldn't. In particular, are there security issues related to the use of message passing for drivers...

Also, QNX has never seen usage as a general purpose OS on a PC.

It has. QNX 6.2.1 offered a full desktop environment. I ran it as my primary OS for three years (2003-2005) while working on a DARPA Grand Challenge vehicle. The vehicle itself ran QNX, and development was also on QNX. An early Firefox and Thunderbird both ran well. The Eclipse IDE ran. All the command-line GCC tools ran. It worked like a typical UNIX/Linux system, but with more consistent response. No swapping. I could run the real-time vehicle code while compiling or web browsing and the real-time code. That consistency in response time made QNX a nice desktop OS.

It disappeared on the desktop after Blackberry took it over and made QNX closed source again. (For several years, all the source was online. Then one day Blackberry took it down, with no warning.) All the open source projects then stopped supporting QNX. QNX development is now cross-compiled from Windows.

With a small microkernel with a good track record, there's no churn. There's no new kernel every week. The QNX kernel had an update once a year or so. This is a big win when it controls your nuclear reactor.

I mean, there's no churn in "the thing called your kernel", but that doesn't mean there isn't churn in the TCB, which is what you care about in security design. There's no reason the TCB in a microkernel would necessarily be any smaller than that of a monolith.

> With a small microkernel with a good track record [...]

https://media.ccc.de/v/34c3-8730-taking_a_scalpel_to_qnx

I'm sure that QNX's ASLR would be fixed pretty quickly if only it wasn't closed source (a massive shame, AIUI Blackberry aren't even really using QNX for anything seriously, it's used in a few car stereo systems, so it's basically "abandonware" at this point).

Well, in case of WireGuard if used as user space app, you would need to copy every invalid (by WG definition) packet from kernel space to user space to discard it instead of dropping it in kernel space. You would need to copy a lot of packets, that do not need to be copied to user space, instead of just processing them in kernel space.

Copying of recent data is cheap. It's already in cache. Fear of copying in microkernels comes mostly from Mach, which was decades ago.

You need to make context switches which are not cheap.

Sure. It is middleware in the same sense that an encrypted filesystem is middleware.

If you like that, you'll love FIPS kernel cryptography!

"He pointed out that Zinc cannot support hardware cryptographic accelerators, something that Donenfeld regards as a feature. "

Why is not supporting hardware acceleration a feature? Or is the objection to something more specific having to do with currently-available accelerators?

The idea is that hardware acceleration is a niche feature, so it should be kept out of the core, low-level crypto primitives, and then a higher-level API can be built that allows dynamic dispatch to an accelerator if available:

https://lkml.org/lkml/2018/8/3/15

> A very large majority of in-kernel crypto users (by number of call sites under a very brief survey, not by number of CPU cycles) just want to do some synchronous crypto on a buffer that is addressed by a regular pointer. Most of these users would be slowed down if they used any form of async crypto, since the CPU can complete the whole operation faster than it could plausibly initiate and complete anything asynchronous. And, right now, they suffer the full overhead of allocating a context (often with alloca!), looking up (or caching) some crypto API data structures, dispatching the operation, and cleaning up.

> So I think the right way to do it is to have directly callable functions like zinc uses and to have the fancy crypto API layer on top of them. So if you actually want async accelerated crypto with scatterlists or whatever, you can call into the fancy API, and the fancy API can dispatch to hardware or it can dispatch to the normal static API.

I guess they don't count CPU instruction based accelerators like AES-NI as accelerators?

They do not. The difference is the same as having a BLAS implementation that uses the SIMD units to their potential and one that supports a dedicated sparse matrix multiply ASIC connected via some high bandwidth bus. In the first case the caller doesn't have to worry about the implementation details it is just a performance optimization. In the other case you have to use an API that has to be more cumbersome to deal with multiplexing the accelerator and moving data.

Correct

That makes sense. Thanks!

Off topic: Does anyone have a good tutorial on setting up Wiregaurd on a cloud server to act as a VPN? I’m currently using Algo from Trail of Bits, which is great, but takes a lot on control out of my hands through its Ansible scripts.

Others have posted great links. But one great aspect is how simple it is. The hardest part is getting a kernel module that works with your kernel. In my experience, the best way to do that is to build it. And that's the hardest part in getting WireGuard working. In Debian, that means using the latest stable release (at least) with the latest kernel.

Once you have WireGuard working, creating tunnels is utterly trivial. For a toy implementation:

    peer 0 with IPv4 address 1.2.3.4
    
    # ip link add dev wg0 type wireguard
    # ip link list
      [see wg0]
    # wg genkey | tee privatekey | wg pubkey > publickey
    # mkdir wg
    # mv privatekey publickey ./wg/
    # ip address add dev wg0 10.0.10.1 peer 10.0.10.2
    # wg set wg0 listen-port 51820 private-key ~/wg/privatekey
    # ip link set wg0 up
    # wg
      interface: wg0
        public key: 0GS...0U=
        private key: (hidden)
        listening port: 51820
    # wg set wg0 peer IlC...QI= allowed-ips 0.0.0.0/0 endpoint 6.7.8.9:51820
    
    peer 1 with IPv4 address 6.7.8.9
    
    # ip link add dev wg0 type wireguard
    # ip link list
      [see wg0]
    # wg genkey | tee privatekey | wg pubkey > publickey
    # mkdir wg
    # mv privatekey publickey ./wg/
    # ip address add dev wg0 10.0.10.2 peer 10.0.10.1
    # wg set wg0 listen-port 51820 private-key ~/wg/privatekey
    # ip link set wg0 up
    # wg
      interface: wg0
        public key: IlC...QI=
        private key: (hidden)
        listening port: 51820
    # wg set wg0 peer 0GS...0U= allowed-ips 0.0.0.0/0 endpoint 1.2.3.4:51820

"Others have posted great links. But one great aspect is how simple it is."

(generate new keys to manage, create new network interfaces, assign new IPs, run wireguard ...)

I would agree that this is relatively simple but only compared to the other mainstream options (namely, OpenVPN and IPSEC) but it is much, much more complicated than sshuttle[1] which distinguishes itself by allowing you to use any ssh server as a VPN endpoint.

No server side software install is required - all you need on the endpoint is an ssh login.

[1] https://github.com/sshuttle

I think sshuttle is pretty cool. It does layer 3 tunneling over TCP in the right way. Usually when VPNs (like OpenVPN) try to tunnel IP packets over TCP, if those IP packets contain TCP data itself, then you have this problem with tunneling TCP over TCP, and the rate and congestion control algorithms of the two instances of TCP step all over each other, and performance becomes miserable. But sshuttle has done something pretty neat to fix that: it actually parses the TCP in the packets it receives, reconstructs those packets into a normal byte stream, and then sends that over TCP, reflowed, to then be converted back into the original TCP stream on the other end. It's a tiny bit involved, but they made it work well, and most of the time sshuttle "just works."

Indeed. It even has (or at least had in previous versions) a configurable ping-time monitor that traded some bandwidth for interactivity - which is another part of what makes it work so well.

shuttle is indeed magic, but it does require python (and permission to run it) at the other end. That’s almost always there, but there are exceptions (miniaturized docker/vm images or tiny image routers)

My only issue with sshuttle is that getting IPv6 to work with it is a real pain.

That's... not trivial.

Compared to ipsec it is.

There are just 10 lines on each peer!

But I do admit that the kernel stuff can be painful.

However, if it goes mainline, that will be moot.

That's about as trivial as compiling a custom kernel. Possibly less so, as compiling a kernel in Debian/Ubuntu takes less than 10 commands.

Yes. I used the latest version of Debian stable available from the VPS host. In most cases, I just upgraded to buster and installed headers. On one VPS that I didn't want to upgrade, I built a custom kernel. The WireGuard build didn't take long, and was no hassle.

Your distro's wireguard-dkms package is your friend.

That didn't always make the nut :(

https://github.com/StreisandEffect/streisand will do WireGuard.

Official documentation and the arch wiki (https://wiki.archlinux.org/index.php/WireGuard) are good sources

https://www.timdoug.com/log/2018/08/04/#wireguard_macos

https://www.stavros.io/posts/how-to-configure-wireguard/

should be sufficient to help you set it up.

Just finished spinning up streisand to run Wireguard on my home network. This is my second time around with this set up, and it works great for me.

https://github.com/StreisandEffect/streisand

using a cloud server as your VPN? what made you come to that decision? how is that better then your ISP?

In my case, I work for an ISP -- althoug not my ISP -- and the "cloud server" is a small virtual guest (that I manage) in one of $work's facilities. Between $work and AT&T (my home ISP), who do you think I trust more?

Many people have very little choice when it comes to the ISP they use. By setting up a "cloud server", at least they get to decide who, i.e. which cloud provider, has access to their traffic -- and can switch between providers much easier and at any time they wish.

that's a good point, but if you go with any of the major "cloud" providers, I don't think that you would be in a better position.

Many of the isps modify your stream - Verizon used to add a subscriber id (so anyone can track and correlate you despite your best effort). Comcast used to add their messages into incoming html.

Does any major (or minor) cloud provide do that?

The cloud providers havent shown themselves to be scum like some of the ISPs that happily inject Javascript into your http traffic.

might be true today

Comcast connections are metered for most residential users. Additional costs are incurred after 1TB of transfer. Also, home uplinks are generally pretty slow, and the VPN will be bottlenecked by its up speed.

you should be aware that VPN traffic still goes toward your 1TB data cap... actually, it probably uses a tiny more data then if you were not using a VPN...

I think cookiecaper read your earlier message as "why don't you run the VPN endpoint in your home" rather than "why use a VPN at all if you're going to put the endpoint at a cloud service".

What is the status of WireGuard in terms of client device support?

The main advantage of IPsec and OpenVPN is that they are either natively supported by all major OSs (desktop and mobile) or there are apps freely available for that purpose.

Linux, Android and macOS (through brew app) are supported, Windows and iOS clients are in development.

Does anyone know a good Windows client?

I only know TunSafe, which now has finally been open sourced. But it was still controversial software. So any alternative would be nice.

I've never really understood what was controversial with TunSafe more than it was closed source so it was hard for others to review the code, but now it's open source.

As a former fan of WireGuard, I would STRONGLY advise against using ANY WireGuard implementation and that includes the official one.

As it is now, there seems to be only one person in the world who believes that they have the knowledge to determine if a WireGuard implementation is secure or not, and it is the founder himself.

So much for the 4000 lines of code that was supposed to be easily audited and understandable by others.

Avoid WireGuard until there is an outside group that can review the protocol and implementations of the protocol. For as it is now, the founder accuses all third party implementations of being unsecure but without being able to state why (more than it feels he has something personal against them).

It all feels too immature.

Wireguard looks awesome. Any news about a windows wireguard implementation?

There's TunSafe, BUT with caveats: https://news.ycombinator.com/item?id=16515637

TunSafe is Open Source nowadays so most of those caveats are moot.

I still believe TunSafe's interoperability and standing security issues are significant, and that TunSafe's existence and confusion-potential are generally damaging to the WireGuard project, as has been the adversarial position of their developer. That project is going in a number of really undesirable directions from a security perspective. I would encourage folks to stay away from this software.

For those who are after Windows clients, the WireGuard project will hopefully have one quite soon, and of course we're happy to work with interested Windows developers who are working on similar projects with a security-minded attitude.

---

I'm sure the subsequent replies to this message will have plenty of outcry, demands for details, misinformation, and accusations, to bait this into a long sprawling thread. I'd like to preemptively step out of that kind of mudslinging. But I do think it's important to warn users, hence the note above.

> If you decide you'd like to open source it at some point, rather than putting ads on it or selling it like you've done in the past with software, we can talk. But insofar as you're putting users in harm's way and fragmenting the project, I ask that you stay away from these parts. Nobody is interested in insecure software.

> Yet in spite of your to-date brazenness, I'm still willing to work with you if you'd like to turn things around. Shoot me an email if you'd like to talk about open sourcing this work and integrating with the community.

https://lists.zx2c4.com/pipermail/wireguard/2018-March/00246...

It's open source now, rather than full of ads or being sold.

In the mean time, I still have ocserv and openconnect on Ubuntu & Mac so I'm happy.

What are the current security issues with TunSafe?

So far, I have not heard anyone who has found any security holes and I'm active in the #WireGuard IRC channel with 300+ users, where many have looked at the code. There may be some unscrupulous hacker who has reviewed the code and found something but choose not to publish it, but it may also apply to WireGuard's source code.

A security hole in WireGuard's wg-quick that many use to establish the connection is that it allows the .conf file to download and execute programs without asking the user, and this feature is enabled by default.

This is basically a good feature and allows admins to run custom software as soon as the connection has been established.

However, it allows an evil (or NSA-hooked) VPN provider to issue .conf files to infect the user's computer with malicious code because users of VPN services rarely review the .conf files.

TunSafe has the same feature but it is disabled by default and requires Admin privileges to enable it.

I like that TunSafe seems to have more restrictive security settings as default, though it may not be appreciated by hardcore users.

I have no dog in this fight whatsoever and I'm grateful for the work you are doing with WireGuard, but I would like to remark that at the moment the only mudslinging and accusations in this thread seem to come from yourself.

I know you pre-emptively opted out of backing up your accusations but I'm going to ask anyway because otherwise it just seems like you are spreading FUD. What are the standing security issues and interoperability issues? Also, how has the developer been adversarial in his position? I'd genuinely like to know.

I'm using TunSafe every day for a while now. It is great. Thanks!

Without any extra tuning I see much better speeds than OpenVPN - something like 20MB/s vs 5MB/s when transferring between EU and US locations.

> Andy Lutomirski was generally favorable as well, noting that he has tried to carry out some similar changes to the cryptographic code in the past. Support for hardware accelerators should, he said, be built on top of Zinc; code needing that support could then use the more complex API that would be required, and the Zinc implementations could be used as fallbacks when acceleration is not available or practical to use.

This seems like a flawless approach. The Zinc approach seems to be preferred (by those involved) for simple software-only use cases, and the more complex use cases seem to be composed of operations which Zinc could implement.

It's good that they're not just going to plop the thing in there in a degraded state (with probably worse performance [and DoS resistance] than the current out of tree/dkms distributions of wireguard).

> It's good that they're not just going to plop the thing in there in a degraded state (with probably worse performance [and DoS resistance] than the current out of tree/dkms distributions of wireguard).

Yea, indeed, I'm really trying to get the mainline version to have the same performance and security characteristics as the out-of-tree module version.

(And after it's mainlined, the out-of-tree module will only exist as compatibility for older kernels, and I'll have some scripts to automatically extract a mainline kernel into a backport.)

Reply on news.ycombinator.com