Would we still create Nebula today?

Yeah it's still my go-to because it's fully self hosted. Nebula can be too but it uses certs that can expire whereas tinc just uses keys.

And really I've been using tinc for almost a decade and I didn't really see the benefit of changing. It's rock-solid.

With the exception of one thing: I use some central nodes on cloud VPSes and they can access everything. As far as I know a nebula lighthouse can't access any of the clients. So I've been meaning to give nebula another try.

But zerotier and tailscale aren't options for me because they rely on their cloud infrastructure. I only want stuff that's fully self-hosted.

There's a great tinc android client these days too.

> No love for tinc[1]?

I actually looked into tinc, and really wanted to love it because of its simplicity.

Unfortunately though, it seems that the development scene around it has been stalled for several years now, and benchmark reports show that it can't keep up with cloud speed tests when compared with Wireguard:

- Wireguard: 390.3 Mbps

- Netmarker: 369.3 Mbps

- Tailscale: 62.5 Mbps

- ZeroTier: 56.8 Mbps

- Nebula: 38.4 Mbps

- Tinc: 34.7 Mbps

- OpenVPN: 22.3 Mbps

Source: [disclaimer, article written by Netmaker CEO, so implicit bias]

(https://medium.com/netmaker/battle-of-the-vpns-which-one-is-...)

That's totally reasonable, and I agree that using something hosted entirely by a 3rd party makes sense for some use cases. Our reason goes a bit beyond security concerns, in this case. We built Nebula for large scale deployments, and because of that, we have made decisions that lean into that model for hosting.

Our decision to leave lighthouse hosting in the hands of users has one primary rationale: We want users to have complete control their network availability. Any downtime of our service should not impact their network availability. You can even host some of your lighthouses inside of network boundaries to ensure that an internal network functions properly if its connection to the internet is interrupted. Other overlay options may continue to work for some time, but new connections are often not possible, and the network can degrade rapidly.

Relays are are a similar story, but with an additional reason: We don't have to limit our customers' relay bandwidth due to cost. When hosting relays on behalf of others, we would be transiting a lot of traffic, which has an associated (sometimes unpredictable) cost. By letting our customers host relays, they can ensure relay traffic is just as fast as direclt connections.

This is indeed understandable. For me, the fact that I can run my own lighthouses and depend on / trust no third party is the big plus. Both cases (third-party and self-hosted) are in demand.

Probably not much? I tried to set up Wireguard to my home network before traveling for the holidays last year. I was too dumb to do it, or at least in a way that didn't feel fragile. I had Tailscale up with subnet routing set up the way I wanted in 15 minutes.

I think mesh networks start to become more valuable when you want/need various users or sites to be able to talk directly to each other with significant bandwidth, or when latency might be constraining, and start to scale up. To take my own examples, I've got my main Site A which has a fixed IPv4, then I have a Site B which is the primary offsite but still "local" (<1hr drive) backup location with a few hundred terabytes of redundant spinning rust and no fixed IP, and then small sites C through M that also have their own local NAS. All are within a few hundred miles of each other. Everything is managed from A, so WireGuard would be fine for handling all remote access, even if I'm in another country since there isn't really a significant extra latency hit from bouncing through A. But I want C-M to backup to B as well, and in a hub/spoke that would mean all that traffic must also go through A. Obviously if I rented a fixed IP for B, I could then set B up as a new hub and add new VPNs and rules for C-M for B as well as A. But that's already a certain amount of extra work or setting up automation, and if down the road I had a B2 or B3 that'd multiply. Even automation though won't solve the problem of latency if things spread out further, if A was a thousand miles away from some new site, now bouncing through A starts to add a potentially really noticeable latency hit for no reason.

Having a mesh means only the lighthouses need fixed IPs, and then everything can talk directly with no extra work. That is significantly more scalable, adding a new client requires nothing from any of the other clients. And it cuts out unnecessary legs in the trip and the traffic and latency that involves.

Wireguard is awesome and it sounds like you probably don't need to bother in your use case, but if you start providing distributed services not funneling through hubs starts to be worth considering. Particularly if you start caring about one service site going down affecting others.

That's a great question! One of the things I enjoyed during my time at Slack was their willingness to contribute to open source projects. We had similar IP clauses, but asking permission to open source things was straightforward.

The most important concern (IMO), was considering whether we could commit to properly maintaining a project. Before open sourcing anything, you need to discuss how you'll go about managing an issue and pull request backlog, so that people don't come across "dead" projects under your stewardship.

In a high growth startup, I do think something like this could happen again, but as a company grows, there are certainly more layers that can make it difficult to share things openly.

Thanks Harrison, hope you're well!

And some kind of cert renewal tool I guess.