Tailscale SSH is now Generally Available
tailscale.com
tailscale.com
I can certainly see the value of this feature for some orgs, but it seems little scary to me. With this setup, if an attacker is able to compromise Tailscale and add a key to your tailnet, that person will immediately have access to your network AND shell access to all of your boxes, rather than just network access if you use Tailscale with vanilla ssh.
I also send myself notifications any time a failed or successful SSH login attempt occurs by tailing the ssh service with journalctl. When I last tried Tailscale SSH, it didn’t log anything to journalctl and so my self-notification via journalctl method did not work.
I use journalctl to follow the log. My NixOS module for it is here: https://github.com/heywoodlh/nixos-configs/blob/master/nixos...
If that isn’t clear, let me know and I can send my Ansible example :)
EDIT: To be more precise I set up the monitoring service as a systemd service
that's why tailscale has https://tailscale.com/kb/1226/tailnet-lock
> that person will immediately have access to your network AND shell access to all of your boxes
it would be extremely weird and negligent to deploy Tailscale at a company and not have any ACLs.
ACLs still come down to "just trust tailscale is working as advertised." And while I generally do (I'm a happy user), if that last few years have shown anything, for the vast majority of companies/products, it's not if you get compromised, it's when. Given that I'd prefer to have multiple distinct layers between the internet and my boxes.
Though definitely see the value in terms of being able to easily tie SSH access to ACLs and your SSO provider -- as with all things security it's a trade off between ease of use and locking everything down.
One of my former gigs used a competitor (also a startup) that was 2x the price and offered way less flexible setup but our “architect” rushed it bc of the way he was. Other more established competitors were even more expensive
Following the adage "if it's for free, you are the product", what is going on behind the scenes? Are they providing their services as a giant honey-pot to sniff on traffic?
> TL;DR: Tailscale’s free plan is free because we keep our scaling costs low relative to typical SaaS companies. We care about privacy, so unlike some other freemium models, you and your data are not the product. Rather, increased word-of-mouth from free plans sells the more valuable corporate plans. I know, it sounds too good to be true. Let’s see some details.
So it's a weighed choice between "if something seems like it's too good to be true, it often is", and "the explanations they give make good sense, and it's a way of doing business that some ethical company could choose to take".
We probably won't know in the short-to-medium term, so we'll have to take their word for it..
But I must admit, their products look pretty impressive. I'll have to have a closer look at them.
Ah, interesting, thanks. That would indeed make it a lot less costly. I would need to dive into it to get a better understanding how their service works.
Would you happen to have some good resources you found useful?
Note that the private key never, ever leaves its node. This is important because the private key is the only thing that could potentially be used to impersonate that node when negotiating a WireGuard session. As a result, only that node can encrypt packets addressed from itself, or decrypt packets addressed to itself. It’s important to keep that in mind: Tailscale node connections are end-to-end encrypted (a concept called “zero trust networking”).
Tailscale SSH is quite interesting because it'd require Tailscale authentication. So it would segment SSH access off, and makes SSH access also generally available to all clients utilizing Tailscale, regardless of host OS.
I checked, and it doesn't seem like Headscale supports SSH access.
Tailscale gives you authorized connectivity between hosts, and DNS; won't it be sufficient to run plain sshd?
(If wireguard-key-level auth were sufficient, even rlogin or netcat would be enough, because the transport is encrypted already.)
It means you can configure/activate/deactivate peoples access centrally too.
But now I need to trust these random host keys, instead of a key signed by my SSH CA...
Anything signed by the SSH CA will work for logins.
To deal with the “untrust” issue it’s normal for operations with an SSH CA to rely on (very) short-lived certificates, meaning often issued and valid for < 24 hours (it’s configurable, I’ve seen this be as short as 30 minutes).
Smallstep wrote a summary here which is pretty good —
So you want a way to get rid of long-lived SSH certificates, instead authenticating users with your corporate single-sign-on system then issuing them a temporary credential?
And presumably you've got some audit logs, so you know who connected to what, when and why. Perhaps a familiar command line tool, that makes temporary credential rotation easy for users? Perhaps some paperwork to hand to your SOC2 compliance auditors?
I mean, this is sounding a lot like tailscale ssh, teleport, and suchlike...
Tailscale comes with an additional SSH server, which runs in parallel with your other SSH server. It does use Wireguard keys directly, so effectively you don't need to manage keys.
Additionally, this SSH server is implemented in userspace, so it won't (can't) interfere with anything else on your system (like your other sshd).
So for example you could have instances tagged with {users} that cannot ssh to certain boxes, but other users tagged with {admin} that can. All these users can be part of your tailnet.
- Wireguard keys for authentication and encryption.
- Tags for authorization to run a remote shell.
SSH to a Tailscale machine
USAGE tailscale ssh [user@]<host> [args...]
The 'tailscale ssh' command is an optional wrapper around the system 'ssh' command that's useful in some cases. Tailscale SSH does not require its use; most users running the Tailscale SSH server will prefer to just use the normal 'ssh' command or their normal SSH client.
The 'tailscale ssh' wrapper adds a few things:
* It resolves the destination server name in its arguments using MagicDNS, even if --accept-dns=false.
* It works in userspace-networking mode, by supplying a ProxyCommand to the system 'ssh' command that connects via a pipe through tailscaled.
* It automatically checks the destination server's SSH host key against the node's SSH host key as advertised via the Tailscale coordination server.
Highly recommended.
I have been using Cloudflare's cloudflared tunnels. It was great for tunneling ssh traffic behind firewalls. And it starts free.
[0] https://developers.cloudflare.com/cloudflare-one/connections...
The equivalent http(s) side of things from Tailscale would be Tailscale Funnel [0], although it's incomplete since you can't BYO domain to TS Funnel.
In essence, CF Tunnel = Tailscale Funnel w/ BYOD + Tailscale SSH
> In essence, CF Tunnel = Tailscale Funnel w/ BYOD + _Tailscale SSH_
> CF Tunnel = ... Tailscale SSH
Secondly, it has clients (on iOS it is called Cloudflare One) which can act as a VPN service as well. You can access any IP addresses (if you setup the vpc correctly [0]) directly accessible to cloudflared daemons.
[0] https://developers.cloudflare.com/cloudflare-one/connections...
I’d be pleased to be proven wrong (and jgrahamc is def ITT) as I use a bunch of CF services already and it would be great to have one less PaaS in my life.
I assume the crypto is unbreakable for outside parties that just sniff the traffic along the way.
But what if Tailscale gets hacked? Are my keys available there for someone else to connect into my network? How hard would it be for the hacker to add their own machine to my network?
About the question about Tailscale being hacked or injecting hosts in your network. With the default configuration yes. There is Tailscale lock https://tailscale.com/kb/1226/tailnet-lock In which you need to sign the devices participating in your network. The signing keys are in your device.
If you use Mullvad VPN you need to sign the Mullvad exit nodes.
Just make sure to have more than one signing device in case something happens to your main computer.
if tailscale got completely owned then obviously an attacker can fuck with the keys, which is why they have this feature: https://tailscale.com/kb/1226/tailnet-lock
Are people just doing this with time-based ACLs within the same tailnet? Curious if there's something more obvious.
> This traffic is rerouted to an SSH service inside the Tailscale daemon instead of to your standard SSH server.
Was their sshd code audited? This is a lot of trust to use different sshd imho.
Are they talking about something different?
Every engineer in every other field of engineering knows exactly how to order things, and has access to a properly organised budget for doing so. You need electronic components? Custom-machined parts? Raw material like metal sheets? Nuts and bolts? Aluminium extrusion? Of course you pay for it - and the organisation's purchasing procedure is something you learn in your first week.
We in the software industry, on the other hand, get free operating systems, free compilers, free IDEs, free databases, free libraries, free documentation and training, free support - pretty much free everything. So you can get surprisingly far in the industry without ever learning the difference between a quote and a proforma invoice - or how to explain to the boss why we should give JetBrains $50k per year in terms she'll understand and approve of.
As such, when the people gifting us and our employers free stuff add a paid tier with features we want, but they charge money for it, it's an almighty inconvenience.
I'm all for price discrimination, but SSO is the exception, because the alternative is too expensive.
So selling this fundamental, foundational security service as a top tier premium value add is disingenuous, and either deceitful or greedy.
And frankly if you're overly cynical about it; it's a lot like streaming in the sense of why am i forced to pay 10 vendors for the same library? There's a data ownership issue, why am i forced to let them own my user data on their cloud unless i pay for the privilege of owning my own data?
1. SaaS providers need it so they don't store your creds. If they don't store them, they can't leak them.
2. You need it because 1.
3. Nobody needs SSO any more :-) Actually you only need OpenID Connect which shows up as "Sign in with" or "Continue with", which -- if coupled with a domain name validation on your canonical user ID -- amounts to most of the same value as the complicated SSO / SAML dance without per customer config. It is less work for a SaaS provider to support sign in with than to make an entire auth chain.
My current recommendation to new SaaS offerings is OIDC plus magic links as a fallback. (Many SaaS go a very long way with just magic links, those plus a domain name in email address check can also tie employees to a company, regardless of the company's IdP.)
All that said, SCIM and group-to-role sync, etc., should be EXTRA. You need extra moving parts, and enterprises with information barrier or other compliance or regulatory obligations are thrilled to pay for this.
I knew of https://sso.tax (which we are not listed on but I did include in my blog), but didn't know there was another website too!
Respect for pushing that through the org!