So while also implementing Tailnet locks and other security measures to constrict traffic flow, I'd also consider going a step further by only permitting server or resource access based on client certificate validation (in other words, a client that's missing a trusted certificate is rejected from even attempting to initiate AuthN); that way even if your Tailscale network is compromised somehow, untrusted clients and endpoints can't make inroads into your infrastructure as easily.
Is that a gigantic PITA to implement? Oh heck, you betcha it is, and I doubt 99% of folks need to go that far with their homelabs or home services. Still, that'd be my approach to zero trust - trusting Tailscale only so far as enabling virtual networking, but not blindly trusting traffic coming over that network at any point.
I use my own self-hosted Wireguard VPN server. I agree with a lot of what you were saying about client certificates etc. And I plan to eventually do that sort of thing on some of my services in my own Wireguard VPN too.
But in terms of Tailscale, if you are going to set up all kinds of client certificate things that will take a lot of time and effort, why not self-host Wireguard also?
Setting up a Wireguard server is super simple. The only couple of things that complicate it a tiny bit is opening up a port for it for inbound connections if you host it from your home connection rather than a rented server, and managing the Wireguard public keys that are allowed to connect.
But if you are going to do a whole client certificate setup on top anyway, the work of setting up your own Wireguard VPN is small in comparison.
Unless like OP your ISP has put CGNAT on you.
Already do! I tried Tailscale initially, but ultimately decided to put in the effort of a proper Wireguard setup. It's how my personal devices always get back to my home LAN, and then exit to the internet; it's also how I make sure every DNS lookup hits the Pi-Hole, for domain blocking wherever I am.
I emphatically recommend learning WireGuard (and to a lesser degree, VPN Concentration) when practical and possible. Until then, Tailscale is an excellent product.
I run Wireguard on a VPS and route public traffic with it over Wireguard to my home machine.
Are you saying my ISP must not be CGNAT or else it wouldn't work?
See earlier in the comment where I said:
> opening up a port for it for inbound connections if you host it from your home connection rather than a rented server
Although I can see how it might not be clear that in the end where I’m mentioning CGNAT I am still specifically talking about hosting the VPN server from your home connection.
Tailscale addressed those issues.
It’s encrypted from client to VPS, then from VPS to home. The VPS sees the traffic inside of tunnel. That’s the first problem.
There is slacks nebula and other options that are completely self-hosted from the start.
Feels like such a weird hype around tailscale.
Tailscale is an excellent piece of software, provided it's implemented in a way to emphasize security, and not weaken it. In OPs case, being used as an accessibility aide to a system that couldn't be secured any other way while preserving external access (in their case due to CGNAT) was an excellent use of Tailscale.
I do think this simplicity is exactly what contributes to those weird and non-standard configurations.
This is why I am confident I will always have employment in IT. As I make things simpler for others to use, they in turn will find new and innovative ways of making my eyes bleed from cursed workflows that once again require professional intervention for simplicity, efficiency, and security.
Tailscale is based on Wire Guard, isn’t it? Now there’s a piece of software that truly made VPNs simple. I have a tunnel back into my LAN by way of an EC2 instance and all it took was two super simple config files on each machine.
Tailscale simplifies: authentication (including OIDC), authorization (via ACLs), DNS, NAT piercing. All of that is not obvious or easy for someone without deeper expertise.
Of course there are tons of alternatives even if you are behind CGNAT. Nebula is but one.
Imo they don't charge all that much relative to their value, depending on who you're asking.
That's not what Zero Trust means, at all.
That being said, the core concept of ZTA is that no user or device should be trusted by default. So yes, my statement is still generally correct even if it’s not how the term is often or commonly used.
This stuff was obvious and standard in the 80s-2000s. It’s only in the last 15-20 years that it became acceptable to get updates shoved down your throat.
Service providers can cut off your access any day.
Software providers cannot unless you’ve given them a live update channel direct to your env.
You emphatically cannot trust vendors, suppliers, users, software, systems, or governments. Ergo, your infrastructure should be built with an appropriate risk assessment in mind, and have proper safeguards in place where feasible. That's just good OpSec.
We took the opposite approach with NetFoundry. (1) We open sourced the code (https://openziti.io/), (2) we built in PKI with private keys generated at source and destination so that even if traversing NF hosted data plane, we CANNOT decrypt traffic, (3) mTLS everywhere, (4) ability to bring your own PKI, and more.
Can you clarify?
Even if an attacker such as the government runs the coordination and relay servers, and the IdP, they will not be able to decrypt any traffic in tailnet.
The secret keys remain on device, and traffic is end to end encrypted. There is no mechanism in the client agents to send out the secret keys. The coordination server receives the public keys and metadata.
Please clarify or revise your comment!
This is where I'm the most curious on what Tailscale will do next. So far all their products seem to contrast at the IP level, but for enterprise use cases there's a real need for application level protections as well. Cloudflare Access is a great example of what I mean.
1. Maintaining high availability
2. Dealing with patches/upgrades
But I'm also really curious how likely a self-hosted instance is to be an attack vector potentially more dangerous than using something like Tailscale.
In my experience as a poor sysadmin (as in, bad), you don't /need/ HA for Headscale because the clients are pretty resilient. I've had my instance go down for a little bit and it's fine. Stale and new connections aren't, obviously, but it will work well enough that you won't realize Headscale itself has gone down until a while after it did.
I do have enough trust in their client that's installed on my machine to believe that it's not actively malicious. I do trust that I can find my other devices, and trust tailscale to keep a list of them, and not randomly add other devices that I don't know, but I don't have perfect trust of that. All my internal services are still E2E encrypted over the Wireguard link; They run HTTPS with an internal cert authority. There's not ports open on them that shouldn't be, and while it's possible that one of them still gets popped, it's much less likely.
I basically just see Tailscale as an auth paradigm for managing wireguard keys.
Just hypothetically, what if an intelligence service records your encrypted traffic and also happened to get AWS to mitm your communication with the tailscale key distribution server?
Doesn't really matter if most of your traffic doesn't use their infrastructure if the most important parts of it do.
https://tailscale.com/compare/wireguard
My understanding is that (in theory) the only way this is possible is if the attacker introduces a new node and then connected to other nodes that are in the tailnet. What you're suggesting is that a single node that is connected to the other nodes gets compromised, but this isn't possible without already being able to compromise that specific node. Alternatively, if someone hacks Tailscale itself, the only way they could get access to any nodes would be to add their own node, but if you have alerting set up you would know and you could shut down the attacker.
Instead, I have a VM running on a cloud provider that I SSH to from an OpenBSD box inside my home network. The SSH connection establishes a reverse SSH tunnel. This opens a port on the cloud VM to tunnel to my OpenBSD sshd port.
With the reverse proxy to my home OpenBSD box established, I can use the SSH jump box option, -J. I connect to the cloud VM and "jump" through the tunnel to the OpenBSD box at home. You can even specify multiple jumps if I need to connect to another machine in my home.
I can also set up a local tunnel through that jump for things like connecting to my Home Assistant server from my remote laptop or phone.
I only have to trust my cloud provider.
My use case for tailscale: have an SSH (or other) connection to my home server while working from home. Drive to a coffee shop, register on their network, and continue using the same connection. (Or hotspot, if I'm somewhere without Wifi.)
The IP address of my server does not change. When at home, the packets do not leave my home network. When out and about, they do.
It's magic to me. I set up a sophisticated (read: overkill) SSH tunneling setup previously, using Match rules in .ssh/config to autodetect the network I was on so that `ssh myserver` would always go via the correct route. But my connections were still interrupted broke when I switched, and I'm not good enough at networking to do any better.
(I guess this is what Wireguard is for? I could access my server via a fixed IP address on my machine that goes to a tun device, and that would send the packets to the actual server if nearby otherwise hand off to the carrier pigeons? Is that what the tailnet is doing? I don't understand how packets get intercepted by tailscaled, though I do see a tailscale0 device. Is that just a vanity license plate version of tun0? Why does `ip route show` give me only routes through my actual devices, then? Never mind, this isn't a helpdesk. I'm just getting old and stupid, I think.)
The tunnel is on localhost only. The VM has a static IPv4/IPv6 with DNS.
Connecting the SSH tunnel from my home is stable as well as connecting to the VM remotely.
I do appreciate Tailscale and Wireguard. I was more responding to the fact that I don't have to trust any provider here, other than the one keeping my VM running.
Also, there's tmux for preserving sessions.
Everything on my home network is set up as if it were public-facing.
By "as if it were public facing" I assume you mean locked down as much as possible using either router or host-based firewall rules?
Let me explain what I mean by low maintenance...
I was a very early containerization adopter and set up a company and also my home network using Docker around 10 years ago. I chose Docker because I thought it was reasonably polished and was the future of deployment. Even though the landscape keeps moving with changes in Kubernetes, Helm, Rancher and stuff like that, the actual Docker part hasn't changed in 10+ years so I haven't had to change my setup for a decade. Low maintenance for me is software that can be left mostly untouched (except for minor updates) for a long time and I judge that based on the project's future, which for me is partly judged from a project's polish.
Every time I tried WireGuard in the past, it didn't seem so polished. I don't want to waste time learning something that could go away. On the other hand, not only did Tailscale look pretty well set up, it was pretty much click and run which means that even if it were to fail, I would have not lost any time learning much about it.
So low maintenance for me is "get the most out of as little work as possible" and choosing Tailscale was the decision to achieve that. So given that I've been using Tailscale for 1.5 years with near 0 amount of configuration and so far, no real downtime, it is adequately low maintenance.
That's Wireguard, I have the same, just Wireguard + VPS, everything I want available that is. I don't put every PC on my home network on the VPN, I could though, pretty easily.
edit: okay, CGNAT
Not just CGNAT but not having _any_ external ports open can be a beautiful thing. I used to have an ssh port (not on the standard 22) and the amount of auth attempts back then was insane. I now have a full firewall zero open ports but, thanks to tailscale, I can still safely access my machines while not being at home with zero unauthorized attempts.
And since I am a security person, I use the tailscale lock feature so not even tailscale themselves can add nodes to my network. Even if they had a breach.
I am a very happy customer.
See xz vulnerability for more details. It’s about not trusting people with any of my ports/software (directly).
> I use the tailscale lock feature so not even tailscale themselves can add nodes to my network.
Also, it seems their infrastructure runs on AWS, not exactly confidence inspiring from a censorship/privacy risk standpoint.
I think tailscale also doesn't provide transient quantum resistance. Wireguard traffic can be made quantum resistant with a PSK. I fail to see why one would use Tailscale over just wireguard other than for "convenience" reasons which are almost never good reasons if security and privacy also matter. Please correct me if I'm wrong with anything, I'm happy to learn.
I do verify every node’s key. That’s kind of the point of tailscale lock unless I am missing something.
> Also, it seems their infrastructure runs on AWS, not exactly confidence inspiring from a censorship/privacy risk standpoint.
I don’t understand what censorship has to do with a personal home network?
Privacy on the other hand, is fair. For my usecase this is a home network I am not that concerned that they know what devices talk to what devices. Yes they know my ip address but that’s not valuable since it’s all defended by the tailnet lock.
> I fail to see why one would use Tailscale over just wireguard other than for "convenience" reasons which are almost never good reasons if security and privacy also matter. Please correct me if I'm wrong with anything, I'm happy to learn. Direct access to my network being limited behind tailscale with a requirement to be part of my tailscale network signature satisfies my requirements for no one else’s access to my network at all. And only if I am away from home does any of my traffic pass through a relay.
Tailscale has more device support than any wireguard apps than I know of. I don’t believe wireguard has Apple TV support, but tailscale does.
I am not the only member of my family either, including them in this network with the simplicity of tailscale’s apps is also important.
I mean, yeah, if you unfortunately have to deal with CGNAT, then you gotta do what you gotta do. But other than that, what's the issue with self-hosting Wireguard?
User simplicity. I am not the only one on my home network which I want to be able to access some parts of the things I build.
Device support. I appreciate that tailscale has gone out of their way to bring tailscale to even more devices than even wireguard supports. Namely apple tv, wireguard does support iOS but doesn't seem to currently support apple tv or maybe just my version of apple tv.