Tailscale ate my network (and I love it) | Hacker News Reader

Tailscale ate my network (and I love it)

smackeyacky.blogspot.com

273 pointssmackeyacky206 comments

I've never used Tailscale, but I want to highlight specifically for working with AWS, you could consider using AWS Systems Manager to access machines that are in private VPCs[1][2]. This has the advantage of reusing the same credentials used already for AWS, as well as being able to further restrict exactly what can be done with them.

[1]: https://aws.amazon.com/premiumsupport/knowledge-center/syste... [2]: https://medium.com/hackernoon/ditch-your-ssh-keys-and-enable...

There's also a new port forwarding parameter on session manager so you can easily connect to a private RDS via a bastion host.

https://aws.amazon.com/about-aws/whats-new/2022/05/aws-syste... https://docs.aws.amazon.com/systems-manager/latest/userguide...

Oh wow, I’m still using ssh tunnels through the bastion. Gonna try this next week, thanks!

I’ve used Twingate for this and found it very easy to setup and use. No idea whether experts consider it a good option.

We still need bastion hosts to connect to RDS, but they're no longer accessible from the internet since we use SSM.

`ssh aws-bastion` invokes a ProxyCommand that:

1) finds an actual bastion host by tag using ec2 describe-instances

2) generates a short-lived ssh key and adds it to the ssh agent

3) sends the temporary key to the bastion using ec2-instance-connect send-ssh-public-key

4) starts the SSM/ssh session using AWS-StartSSHSession

Since it's SSH, we can port forward, use multiplexing, etc. We use Google as the IdP, via AWS SSO. Bastions periodically sync users from a Google group.

Someone on Reddit posted a shell script[1] that does all the connection setup, key-sending, etc that's a useful base to work from.

[1] https://www.reddit.com/r/aws/comments/df6uip/ssm_tunnelling_...

Why do you need steps 2 and 3? I’m able to use SSM with agent without any SSH keys. Also proxying to RDS. Only AWS credentials are required.

Yeah, SSM supports just opening a terminal on the remote host without ssh. We started with regular ssh bastions though so I just stuck with that. They're just no longer routeable from outside.

This may be of interest to you.

https://aws.amazon.com/about-aws/whats-new/2022/05/aws-syste...

It still requires the bastion host though, right? In which case I'd prefer to just use ssh since it's more familiar.

Also, intellij's built-in database tools support ssh tunneling, it 'just works' with the ProxyCommand method.

Yeah, just removes the need for SSH and keys mostly.

Tailscale is insanely easy, simple, and pleasant to setup/manage compared to AWS anything. The thought of using a cloud-specific solution is kind of disgusting in comparison.

If Tailscale-the-product ever goes rogue or evil, I can always self-host wg or a full on tailscale-equivalent mesh myself. I sleep well knowing this.

There's even a mostly feature complete open source control +DERP server on headscale

Thanks for mentioning it by name. Yeah, headscale is very badass.

https://github.com/juanfont/headscale

If you were looking for an open source, self-hosted implementation of the Tailscale control server (as far as I know, that is the only portion of tailscale that tailscale keeps proprietary, and this is the best open source implementation of it).

Edit: wow, this project has really grown from when I last saw it. It is able to configure the vast majority of tailscales base featureset such as ACLs, magic DNS, taildrop file sharing, and so much more. Incredible.

Thanks for pointing this out, going to give it a whirl! Does this solve the thing about having to login using one of Google, Microsoft or Github accounts?

You can use regular tailscale with an e-mail account.

no, you cannot; you must use a SSO provider.

There is a big “Sign in with Email button” after installing the app in iOS.

Edit: Oh no, indeed when you want to sign up you need an sso provider indeed! This is what they say:

Can I sign up with an email address?

We don’t support sign-up with email addresses. By design, Tailscale is not an identity provider: there are no Tailscale passwords.

Using an identity provider is not only more secure than email and password, but it allow us to automatically rotate connection encryption keys, follow security policies set by your team (e.g., 2FA), and more.

> Using an identity provider is not only more secure than email and password ...

It's also a pretty effective way to limit the amount of tailnets a user can have.

Headscale seems to have experimental support for OpenID, so if you plonk it down next to a simple OpenID server for authentication you should be good. You should be okay with anything from SimpleID to Keycloak as long as it supports the right endpoints.

I have no idea how the official clients will deal with that, though, but I've never used tailscale myself.

> plonk it down next to a simple OpenID server for authentication

Could you please elaborate on this solution? I'm not sufficiently knowledgeable about OpenID to quite understand what you mean, but I'd like to avoid any of the mentioned SSO providers, as they're all blocked on my systems for personal use.

Added: Found these as per mention in your post:

[0] https://openid.net/connect/

[1] https://simpleid.org/

[2] https://www.keycloak.org/

... so I assume you mean that I could install one of [0-2] along with Headscale [3] to get the similar effect of installing Tailscael, just without those annoying SSO providers? I will see if I can find the time for examining that solution. Anything that can keep MS and Goog away is most welcome

[3] https://github.com/juanfont/headscale

Yes, with your open OpenID server you basically become your own SSO. I've set up a Keycloak instance for my self hosted stuff and now I can add 2FA to almost any web self-hosted service without the service even needing to have support for it.

Keycloak is quite a complicated system to configure, though, there are easier alternatives out there. If you're just trying to get anything up and running, something simple like Authelia may be better for your use case (disclaimer: I've never tried it, but it seems light weight and other people online seem to recommend it).

> as far as I know, that is the only portion of tailscale that tailscale keeps proprietary

Also the iOS client, which is hard-coded to only use the first-party control plane, so you can’t use headscale if you have iOS clients :(

Shower thought:

If I didn't have the goal of buying a house in the Bay Area, I would totally try to find a job with Tailscale.

If you have to take a job you don't like to be able to afford the house, be aware that nothing changes once you own the house. Most likely you'll still need the job to afford the running costs of the house.

There's no "I'm free once I'm a home owner" thing.

> There's no "I'm free once I'm a home owner" thing.

I finally signed up to an account to HN to say emphatically that it's the opposite.

Owning a house is a huge responsibility that takes up the vast majority of my time, and it's a millstone around your neck if you ever think about moving somewhere else.

While this can certainly be the case for some houses, it really depends on the place and its condition when purchased.

That very much depends on where you live. I know what you mean and I partially agree with it but it's not universally true.

I was happy to sell my house and trade the loss of equity (and the last 10 years has been outrageously exceptional to the usual appreciation of property) for the freedom that living in an apartment gave me. Leaving for a month? Ask your neighbour to water the plants and walk away. 10-20 hours of maintenance and upkeep each week? Now it's 0. Constant accumulation of tools, devices, and products? Not needed.

I seem to be an exception, but I have no need to define who I am by my housing, nor do I need the security of owning my own house. I have also lived on three continents, so I appreciate the ability to pull up stakes and move with very little fuss.

If I was living by myself or just with my wife I would stick with an apartment. The kids spend so much time in the backyard, and it is hard to find an affordable apartment that has enough space for all of us to live, let alone also WFH that my wife and I are both doing. Plus, how do you build projects without a garage and outside area to work on things?

> 10-20 hours of maintenance and upkeep each week?

What in the world? This doesn't even come remotely close to passing the sniff test. Is your idea of homeownership like, constant remodeling or something?

I do zero hours of maintenance/upkeep per week, just like you with your rental.

Is your landlord now doing 10-20 hours of maintenance/upkeep per week for every apartment? Of course the answer is no.

10h is maybe an exaggeration, but I sincerely hope that you're checking your plumbing and all your drainage and maintaining your lawn.

In Sweden, homeownership vs apartments is quite a large difference.

I've seen houses which just grow shrubs over the whole lawn (sidesteps lawn maintenance, I suppose you may have to trim shrubs but its not that bad, really, compared to grass which grows almost overnight).

The other side of keeping up with maintenance - if you don't have time nor want to, hire away. It will eat a significant chunk of change to hire all the professionals your landlord was hiring, the difference being you are hiring yourself versus being dependent on the landlord.

For the DIYer, tools acquisition is definitely a PITA. Hardware stores have a decent selection mostly of what you need, but it would be nice if there were preset of tool maintenance you could order, maybe even save you money over long term buying it all piecemeal.

I'd imagine the real time suck would be for planning/research for it all, if you are used to spending your time coding, playing games, or otherwise amusing yourself, yes for the first couple years you will not have any time for these things as you are acquiring your skillset(s).

But that could be said for going back to school, etc. I think it's pretty valuable to be able to maintain your own dwelling apparatus, personally. It means you'll never be without reasonable shelter, so long as you have some access to raw materials, tools.

Checking your plumbing and drainage? What do you mean? What would you do weekly that involves this?

Lawn maintenance is maybe one or two hours a week at most, and that is mainly because we like to keep it pretty tidy. Then again, we would have needed to do similar maintenance (in the UK) for a rental if we wanted to keep the same standard for the garden.

I don't check my plumbing or drainage. And a lawn is a difference between a house and an apartment, you're right about that, for me it's about 2 hours a month (which I pay someone else to do) rather than uh, 40-80. However I believed we were discussing ownership vs not ownership, rather than apartment vs house.

You didn't do enough research into your house purchase, or you weren't able to afford a well built house.

Houses like anything else have lifetimes, You don't get to be a 200 year old house without having major maintenance done at least a couple times. Buying a poorly maintained, or constructed, old house is a nightmare, if it's bad enough you've discovered why some houses are condemned.

Modern construction often has a longer lifespan and more readily accessible materials, older houses are a mixed bag - some gems that may last hundreds (thousands?) of years, lots of houses that need major repair. Some in the middle too...

Welcome to HN!

Home ownership is as much a comfort as it is a fantasy. There are true perks, eg, you're mostly in control of the regular costs (rent vs mortgage) but it also has downsides, like... basically any hazard and everything location-related.

PS: I feel weirdly kinda honored that you created your account and replied to my comment. Anyway, I'm just being me ;)

Out of curiosity, why are you assuming that not exactly what parent was thinking?

> nothing changes once you own the house

Why? What does this mean and is it actually true? The monthly/yearly cost of owning a house is typically much lower than the mortgage payments. If buying a house in cash, the maintenance usually is low enough to consider lower paying jobs. Needing a high salary might enable remodeling but basic taxes and upkeep are very very different from sale price or the payments to a 30 year loan. In my experience something absolutely changes once you fully own the house.

Recently replaced the roof on my house and the cost was equivalent to a year of mortgage payments. Prior to that, had to replace the AC for about 6 months of mortgage payments. Next big expense is windows, which I expect to fall somewhere between the two.

Totally agree there are some sizeable maintenance costs, I’ve had to do a roof, windows, remodeling, plumbing, all kinds of stuff. It’s still lots less than the purchase price. And I had to pay them on top of the mortgage, it’s not like maintenance waits to start until the mortgage is paid off, right? Paying off the mortgage simply eliminated one money drain for me but didn’t change the other, so my average monthly expenditure went down.

Rentals have exactly the same maintenance requirements as owned property. The fact that you pay for a lease doesn’t magically make the roof last forever.

As a renter, this overhead is baked into your lease. As a homeowner, I can simply tap into home equity to do a major repair at single-digit interest rates over a decade. Something you will never be able to do as a renter.

$15k to replace the sewer plumbing. $9k to replace the sewer line. $9k for a replacement roof....

I mean, yes, if you do things yourself it can be significantly less expensive, but most of the housing stock in the bay area is atrocious - good bones and awful everything else. They were thrown up as quickly as possible in the 50s-70s and so there's always something that needs fixing.

$18k+ a year in taxes...

You should watch tiny home remodeling, its a tv channel (I think?) I was shocked that a couple in NYC bought essentially a cottage in a high-rise for 10 million, then did a half million dollar renovation on it...

In the country-side that kind of money gets you a 12 bedroom mansion with a pool and a view...

I will check it out if it is online.

I do a lot of DIY but there are times when you just want it done and done quickly and professionally. We work for a living and don't really have time to deal with a lot of the DIY incremental aspects that happen until you get very good at any given skill. I just redid one of our bathrooms, but I wasn't going to redo the sewer lines solo let alone mid-week while living in the place.

The problem with HCOL is that even if you own you're burning cash due to the lack of time and very, very high cost of services.

All that seems like scratch compared to the one or two million sticker price, doesn’t it?

Not really. It adds substantially to the 30Y total cost.

That wasn’t the question, right? I think you’re completely agreeing with me and disagreeing with @xcambar if we’re talking about a 30Y loan and maintenance costs. Maintenance costs and taxes start on year 1 and continue forever. So when you finish paying off the mortgage, the substantial loan payments end and go away while the maintenance costs continue. This will be a big change in your expenditure, speaking from experience.

You're bringing a paradox here: "Owning a house is cheaper if you buy it cash, then you don't need a high salary".

But you DO need a high salary in the first place.

And with that big salary, you will buy a NICE house, which will require a lot of maintenance (house keeping, gardening) and taxes.

There is no paradox. Buying a house is more expensive than maintaining it, partly because maintenance is somewhat independent of purchase price.

I think you’re trying to say that an expensive house is more expensive to maintain than a cheap house. That’s true. But the salary required to buy any given house is higher than the salary required to maintain it, for the most part. I’m sure there are counter examples, but on the whole most people who pay off their mortgage experience a pay raise, effectively, which has been my own experience.

Aren't houses in the Bay Area bound to become lower price in the future due to new building being allowed, or subsidies?

Seems like a risky place to invest at this stage.

Treating it as an "investment" and not a "cost" is not really helpful.

The way I usually treat it is: Can I pay off this mortgage in the event I don't want to sell the property?

Negative equity is a thing, sure, but if you were always happy to pay the price then the thing you're buying is worth the price, right?

Sometimes people think of things as having value only if they're relative to something else. But value is value, and value is the price that you're willing to pay.

I keep catching myself thinking this too.

In general, I think it's kind of an interesting heuristic to think about every so often: right now, putting aside practicalities like my current job or where I live, what looks like a cool place to work, even if the specific role was just taking out the garbage.

Right now the answer is "tailscale" and "oxide" for me.

Fly.io also would be on my list

oh yeah good call.

Definitely :)

In the past I also liked sourcegraph, too. Not sure what they're up to nowadays.

We're here :)

You don't have to live in any particular place to work for Fly.io; we're all remote, and we hire all over the world, at the same west coast comp rates everywhere.

its not really that hard to setup amazon system manager. like maybe 2-3 hours of overhead.

the idea that you'd pay a subscription fee greater than aws just to avoid a few hours of learning how to set something up is kinda disgusting in comparison.

Why do you care what other people do? Just focus on yourself.

Resolving to a hostname in n AWS using a tag is eirksome though. MagicDNS on the other hand in Tailscale makes it easier.

FWIW I use both SSM Port Forwarding and Tailscale but Tailscale is easier for both tech and non-tech users IMO.

I wrote a ruby script that does tag lookup for me with an interactive prompt. Took about an hour. Sent it to my team so we all use it. Can't imagine subscribing to a service for something that is a minor devops task.

I’m going to be that guy at point you in the direction of “that comment” on Dropbox

https://news.ycombinator.com/item?id=8863

This was my comment re. AWS SSM and Tailscale just a few days ago.

'Installing the agent client side is no more or less tedious than installing the Tailscale client, IMO anyway.

I made two scripts, one in .Net with a GUI for non-devs to grep a server hostname or tag:name in AWS that resolves to an instance ID for SSH or RDP. And another python script doing the same but without the GUI for the dev team. Works a treat.

But you've already explained why it's a little tedious and now I've documented and understood why. Tailscale MagicDNS does all this nonsense for you. Yeah ok thanks for rubber ducking me I see your point now. :)'

How many devices do you need to keep connected? I'm surprised the free tier wouldn't work for you to be honest

Huh? Why would I use a paid third party service (free tier or not) for something that I know how to use at no additional cost?

I might be misremembering this, but IIRC if your nodes are running on latest Amazon Linux, they already have the ssm-agent pre-installed.

At most you'd need to add some permissions to the instance's role, maybe.

why dont you use zerotier?

Can't speak for the parent commentator, but I gave zerotier a try and ended up dumping it mostly because it was unusably slow on single core Linux VMs, making it not a viable option for connecting lower tier cloud VM options. I believe this is on the list of things they're fixing with their next major version, but that version has been very slow in coming out.

>single core Linux VMs...

i can't say about that specific thing but i've been using this daily 24x7 for the last 2 years now. it connects my 20+ pcs and laptops which are geographically apart but because of zerotier, they are in a local lan.

it does not have SSO, relying on the admin accepting/rejecting connected devices by a single checkbox. this is in comparison to tailscale which uses Oath, meaning you have to create and maintain those accounts as well.

ZeroTier has SSO now, at least for its hosted controller option.

I really like ZeroTier, but the nail in the coffin for me has been that there is no ability to self host a controller, while also using the management web GUI.

Their sales team, when I asked about self-hosing a controller, said it's not necessary because they've never had all the hosted controllers go down, but when I asked about a tweet they sent in May 2020 about their controllers being down, I never got a reply. [1]

My plan was to put ZeroTier on all of our machines and use it as an overlay network that all traffic goes over. But I don't want to open the availability of our network to depending on an external service.

I've all but decided on Nebula, just need to get the deployment worked out. I'm playing with Tailscale right now, and am very impressed. It does have the ability to require MFA on logins that we would like for user VPNs, while still being able to have servers self-authenticate (we respin half of our dev/stg environment every night).

[1] https://twitter.com/ZeroTier/status/1389766385480372225?s=20

> Tailscale is insanely easy, simple, and pleasant to setup/manage compared to AWS anything.

Seriously? What equivalents does it have for Route53 and tons of other services?

I can't tell if this is misleading or pointless.

The post you're responding to wasn't arguing for capabilities, it was arguing for ease-of-use and it was right.

Your comment is pointless and a straw-man argument.

The commenter wrote that Tailscale is easier to use than AWS anything, but it can't even stand near in terms of scope. It's like saying that my calculator is easier to use than Apple anything; useless astroturfing that adds nothing to the discussion.

They do support DNS, to answer your original question. But to this point, it's not useless because someone suggested they use an AWS service to accomplish what the article says was done with tailscale. So, saying tailscale is infinitely easier to set up is a very useful comment. Also true.

No, it's not even true. I really went to Tailscale to see an alternative to AWS suite. Turned out to be a sort of managed Wireguard instead.

Yes, you're right that "it can't even stand near in terms of scope", but the original comment did not argue that, the original comment gave an opinion about the usability (actually "to setup/manage", so more like the administrative efforts) of Tailscale, which is not the same thing as the scope of its abilities. Calling the comment misleading because the service has a smaller scope is then at best misunderstanding the original comment, at worst purposefully trying to derail the discussion.

If your sole use case for wireguard is "I need ssh access to a fleet of EC2 machines" then System Manager is definitely cheaper and easier to setup. Even more so if one is using IaC.

FYI when AWS had their little oopsie with the EC2 management APIs not too long ago, I couldn’t connect to an instance via Systems Manager. I also couldn’t adjust my security groups to enable SSH. I also couldn’t shut down the instance. AWS refused to provide any credits as their SLA only covers when the instances are not publicly accessible.

We're using AWS SSM to allow access from a developer's machine to a locked down k8s cluster in AWS.

It works but performance is awful. Slow connection times and pushing images has slow throughput.

You can also use AWS Client VPN. It’s not as user-friendly as tailscale, and probably more expensive(?), but it works well enough. It’s also pretty easy to provision with IaC—not sure if that’s the case for tailscale?

I'm currently evaluating Tailscale and other VPN solutions as AWS Client VPN doesn't look like it's going to work for us.

AWS Client VPN does not offer any means of unattended configuration or mass distribution. All they offer is a self service portal from which you can download the installable, the profile, and directions. Each use has to manually import the profile into their client.

I'm stunned they're missing this as it would prevent any sizable organization from adopting it.

Yeah it’s not ideal. It’s nice not to have to run any kind of agent, and to use IAM for access control (not that IAM is so wonderful, but it plugs in easily). I agree though it could be much smoother.

Huge fan of Tailscale here too. They solved every single complaint I had with using WireGuard (provisioning, key exchange, IP assignment, ACLs, etc.) and did it in a splendid and elegant tool that just disappears.

One of the few products I recommend enthusiastically.

To be clear, WireGuard seemed to have the right level of abstraction as a tool for others to build on (just like it built on top of the noise framework), and someone like Tailscale ran with it.

The main reason I use wireguard on my personal network is to keep 3rd party clouds out of it. My understanding is that a closed source node that phones home is part of their solution. For my personal case, this is a total no-go.

It's worth noting that the Tailscale client is, almost entirely, open source. It's just the iOS/macOS/Windows client code that's closed source (just GUI wrapping it), iirc[0]. The DERP code is also open source.

Additionally, there's an open source reimplementation of the control plane called headscale, as well. The Tailscale team has complimented it, but of course it's all on your own if you choose to run it.[1]

[0] https://github.com/tailscale/tailscale

[1] https://github.com/juanfont/headscale

EDIT: Android code is actually entirely open, oops!

Wonder why those are closed source if just gui wrapping.

To make it harder for a competitor to pop up and offer their own flavor of the binaries and support. It's just to raise the barrier of entry to others monetizing your project

Maybe because nobody on iOS/macOS/Windows cares if there's closed source code running on their machine.

I do, and wherever all other things are equal, I always choose an open source solution. But if all other things were always equal, I'd be running OpenBSD full time, so...

Being open source is not as useful on iOS though, since there’s no way to verify that what they uploaded to the App Store was actually built from the source that you can see.

How often do you rebuild your system and verify all checksums? What's your threat model?

The point was that you _could_ verify whether a binary package matches the source if you wanted to on other platforms, not that you _have to_ do it all the time. With iOS you don’t have that ability so you’d have to have complete trust in the developers. The threat model is someone publishing innocent source code but sneaking in something malicious in the version they publish on the App Store, of course.

> The point was that you _could_ verify whether a binary package matches the source if you wanted to on other platforms [...].

Which platforms?

From Debian wiki (https://wiki.debian.org/ReproducibleBuilds):

> Reproducible builds of Debian as a whole is still not a reality, though individual reproducible builds of packages are possible and being done. So while we are making very good progress, it is a stretch to say that Debian is reproducible.

Apart from Debian and NixOS, I'm not aware of any other large-scale efforts to make builds reproducible, so this something you have to assess on a case-by-case basis for every single piece of software you might want to download and install. Not IF the checksums match, but IF the project aims for reproducible builds in the first place.

By the way, if you download an iOS app on an M1 Mac, you can inspect its archive freely, including generating and comparing checksums of each file, disassembling the executable, and so on. So while not every iOS app can be downloaded on a Mac (the developer actually has to opt out), and not everyone has access to an M1 Mac, this is not entirely impossible.

> With iOS you don’t have that ability so you’d have to have complete trust in the developers.

This is not entirely true. I also have a certain degree of trust in the application sandbox provided by the OS, and in the app review process. I know neither is perfect but it's not like everything I install runs with full root.

> The threat model is someone publishing innocent source code but sneaking in something malicious in the version they publish on the App Store, of course.

What is your strategy for applications that do not publish any source code? Do you exclusively use software with source available?

Don't get me wrong, you raise valid and important points, but this is a way bigger fish and "if only Apple did X" is just the first step.

> Which platforms?

I wasn’t thinking of reproducible builds specifically, more that you’d be able to disassemble the binary if you wanted to check, because…

> if you download an iOS app on an M1 Mac, you can inspect its archive freely, including generating and comparing checksums of each file, disassembling the executable, and so on.

I did not know this. I thought iOS binaries are always encrypted. Why would they make the unencrypted binary available for M1? And now that they did, why would they bother still encrypting the same binary on iOS? (Do they still?)

> I also have a certain degree of trust in the application sandbox provided by the OS, and in the app review process.

I’m not too worried about an app escaping the sandbox and harming the rest of the system either. More that it could do something malicious with the data it has legitimate access to. For example, a password manager sending all your passwords to a 3rd party, a browser app injecting JavaScript into the pages you open, or a VPN app analysing all your traffic.

> What is your strategy for applications that do not publish any source code? Do you exclusively use software with source available?

Well _you_ were the one who preferred open source, not me. I was merely pointing out that “open source iOS apps” might as well be called closed source, since I thought it was impossible to know if you’re really running that code on your device.

> I wasn’t thinking of reproducible builds specifically, more that you’d be able to disassemble the binary if you wanted to check [...]

Disassembling and analysing a binary (even with source available as a reference) is an *extremely* huge leap from comparing checksums. We're mere mortals. Get real ;)

> I did not know this. I thought iOS binaries are always encrypted.

I've just checked a bunch of iOS apps I have installed on my Mac (Apollo, Blink, Bandcamp, Organic Maps, iSH); ran strings, dyld_info, otool, etc on them, no walls. Maybe only the headers, rodata, linker metadata, etc are unencrypted? Should I keep digging?

> Why would they make the unencrypted binary available for M1? And now that they did, why would they bother still encrypting the same binary on iOS? (Do they still?)

No idea. But the binary being distributed on macOS and iOS is the same; or at least, it doesn't need a separate build/upload, apps published before the debut of M1 just work.

My company actually needs to release an update to an iOS app in the near future, perhaps I can take the chance to compare the artifacts.

> More that it could do something malicious with the data it has legitimate access to.

There are two possible scenarios where this happens: the developer being malicious, and a supply chain compromise.

In the first case, the developer risks getting ostracised by both the community, and the Apple overlords (e.g. for violating App Tracking Transparency). It's also a very risky move, since the traffic from the app can be analysed (e.g. on your home router) at any time, without the app ever knowing; and without e.g. certificate pinning, can also be snooped or MITM'd; this puts a timer/window on it. So in practical terms, for this to be remotely useful or worth the risk/effort, this would have to either be a narrowly targeted attack, and/or a one-off smash & grab. Again: not perfect, but the system has countermeasures.

The latter case is something we (as in: the entire industry) are still trying to figure out. Even with a fully OSS supply chain, it's far from ideal: https://drewdevault.com/2022/05/12/Supply-chain-when-will-we... - and again, in case of the App Store, with a third-party (Apple) review step, there is at least the idea of independently verifying the app's legitimacy.

> I was merely pointing out that “open source iOS apps” might as well be called closed source, since I thought it was impossible to know if you’re really running that code on your device.

You've never asked me, what was my own motivation for preferring free/open source software - and made a bunch of assumptions, not all of which held.

However I do share your view, that under certain (actually, very common) circumstances, software that is nominally free/open, can be considered closed in practical terms: and that, in my book, is rampant complexity. If the code is too complex for me to read and fully understand it, what use do I have from source access? Hence, my actual vote is for OpenBSD; but sometimes you just need to compromise to get stuff done.

For tailscale, I suspect open source is a way to market their software. I guess open sourcing iOS, macOS, Windows isn't worth the marketing advantage.

https://apenwarr.ca/log/20211229 (the gift of it's your problem now)

Well, you can't sometimes or halfway be committed to software freedom, so it's because they only open sourced the other parts for marketing.

So they can make money?

Because if I need to deploy some tech for a company with a thousands of clients I want something what doesn't need my attention for every other client station. I need something robust what would work 99% of times and for a 1% what disbehave I can OR just change the environment so the client would run or drop a ticket to the guys who I am paying for diagmosing these things and after a couple of days or weeks:

Just install a new version which would work OR

Have the understanding how I can change the environment so it would work.

The thing is what I don't need to spend my time on solving these things.

Wireguard is absolutely fantastic for a small network, when you know what you're doing. It's dead easy to setup and configure, you forward some ports, and boom you have a shared network.

I couldn't believe how easy it is to configure. I set it up for work, do a key rotation on a schedule and it's great.

Tailscale I use for home use on my personal laptop + machines and it's fantastic as I don't need to port forward or anything. I changed over from using Wireguard with minimal effort, just changing some IPs over. Probably end up using MagicDNS (their DNS solution), then if I need to change off I'll just change this.

Their free tier is very generous for a single user.

My problem is that Wireguard alone works too well. Years ago I got it all set up, forwarded the one port, and it’s been perfect ever since. I really want an excuse to try Tailscale. Some day. Haha

Excuse is no need to do port forwards.

Having a VPS in the mix removes that hassle.

Is this a replacement for something like Mullvad?

No. Mullvad is VPN to the Internet, Tailscale is a VPN among your own machines.

It seems like any code running in your browser or on your local machine has access to your home network, which was always true, but now your "home network" includes machines in multiple locations, including AWS.

That’s a very interesting perspective and I wonder if that will change attack vector.

One reason to have gateways etc. was to ensure that gateway couldn’t be taken over by software installs etc. Access was inconvenient but it was somewhat by design.

Maybe due to this attack vector shifted from directed access to automatic scanning of ever expanding vulnerabilities.

Now, as services stops being accessible from external networks once again they can be accessed in convenient ways thus bringing the old vectors back. Sometimes even giving false sense of security.

Tailscale has ACLs and then beyond VPNs there's zero trust and BeyondCorp.

Shoutout to ZeroTier, Nebula which essentially do the same thing. Or Netmaker if you wanna go complete open source/self hosted.

Zerotier works at a lower level. This means you can do other protocols than tcp/ip

I don't think that's correct, they both expose TUN/TAP devices so should be equivalent in their encapsulation abilities. I use both regularly.

https://en.wikipedia.org/wiki/TUN/TAP

It is correct because a TUN interface is not the same as a TAP interface. The latter is layer 2.

Tailscale can do UDP also.

"TCP/IP" means TCP, IP, and UDP.

ZeroTier can do other protocols like NetBIOS, so you can use it to run LAN games that don't use TCP/IP.

I tried ZeroTier in the past, and it was great, but didn't have an iOS app. Do any of those do?

ZeroTier has an iOS app now.

Yeah netmaker is really easy to set up

I organically grew my tailscale network and with the recent `tailscale ssh`[0] it has turned my life around. I have no open ports to anything & be it my personal machine in the depths of my closet or stuff on the cloud; everything is seamless connected.

[0]: https://tailscale.com/tailscale-ssh/

For those still using SSH as normal, you can setup Tailscale to accept connections only from Tailscale, and ignore any public internet traffic i.e., restrict ssh access to be only over Tailscale. For example, with UFW you could delete every rule except for the “Anywhere on tailscale0” and “41641/udp” rules.

Exactly how it `turned my life around`.

I can now go to sleep without having to worry about random bots trying to mine crypto on my machines. To add to the goodness, one does not have to worry about either SSH-keys or remember cryptic passwords.

This seems like a ... strange ... comment. Do you usually worry about your ssh keys getting stolen? How do you use git?

FWIW, setting up something like tailscale is remarkably simple (I'm using PHP here to keep it simple):

In sshd_config:

    AuthorizedKeysCommand /auth_ssh %u
    AuthorizedKeysCommandUser nobody

And in auth_ssh, verify that the user is allowed to connect to that server, then look it up on github (my public keys: https://github.com/withinboredom.keys).

If you want to allow any github user you allow to connect various permissions, check out libnss-ato.

These are all 1 or 2 lines of configuration and are not hard. You just have to know they exist.

Thank you for the pointers. Honestly, I had no idea about what you pointed out.

Regarding the strangeness; yes I was lazy and had password based ssh and was in constant fear that someone might pop my box.

I really want to enable tailscale ssh but I often need to ssh using my phone and so far none of the android ssh clients I have tried work properly with tailscale ssh.

I believe they are all based upon variations of the same java ssh library and exhibit the same behavior. They all connect to tailscale ssh using 'none' authentication but after connecting don't display anything which means I can't get the URL tailscale ssh presents to do its authentication.

Edit: I was just able to work around the issue by installing Termux and using openssh in that environment to do my initial ssh authentication. Afterwards my normal ssh app works.

on iOS, at least, the Tailscale app will pop up a push notification for you to authenticate in that case; I'm not sure if the same is true on Android but could be worth checking your notification settings.

Notifications are enabled for Tailscale but I don't get one to authenticate.

I was just able to work around the issue by installing Termux which provides a small Linux environment on your phone. I was able to use openssh in the Termux environment to connect and get the authentication URL. After that my preferred ssh app can connect without issue.

Tailscale seems like a great product however I do not want 3rd party to be able to add a key to my ACL. Running a custom control plane server is possible, but then there is little benefit for me compared to direct wireguard with a central peer on a VPS. If it would be possible to use just the NAT traversal without key management, that would be it!

Curretly I am running a tiny VPS as a wireguard server, but I do not trust it to be part of my network. Therfore I run one wireguard tunnel to be able to access my router (has no public ip) and second tunnel inside the first to connect through the router to my home network.

Theoretically, it should be possi le with single wireguard tunnel if I set a route to home router via wireguard gateway - but I never managed to make wireguard encrypt a packet if it came from the same wg interface. Can anybody help?

I think Tailscale have the right approach by knowing their customer — someone who is happy to have a trusted 3rd party administer parts of their VPN in return for time and cost saving. There are a few here who can't have that, so they instead invest their time into a custom setup with WireGuard which is fine, but for those of us who don't require that level of assurance (there are bigger attack vectors to worry about), Tailscale is fantastic. Quick, easy, and mostly works out of the box.

Tailscale has been a godsend for my team, saving us quite a bit of effort with VPN/firewall administration. There are very few rough edges, and it tends to just work (at least at our scale of a few thousand nodes). We moved over about 8 months ago and have had no issues since. I’ve also moved my home network (RPis, NAS, etc) to their free tier so I can access it remotely.

Some features that are basically effortless and made me choose it over WireGuard and other VPN solutions: easy provisioning, key exchange, IP assignment, ACLs

I recently set up Tailscale, but unfortunately the phone app leaves a lot to be desired battery-wise (it takes up 30% of my total battery usage) so I think I'll be looking elsewhere.

Initially, I had tried setting up Nebula, but I am unable to get a static IP address for the beacon (a requirement for any of these mesh VPNs), hence why I went with Tailscale which acts as a beacon for you. I think I'll try ZeroTier next.

30% seems way _way_ higher than I would expect.

I've been running Tailscale on all my devices for a couple months now, and I haven't noticed any impact on battery life. I just checked my phone (Android) and it's reporting 1% usage.

I would report an issue if you're seeing numbers that high.

I only activate it when I actually need to use it. I don't like running any phone apps in the background, they always suck juice.

I use an iPhone. If you google it, you will notice that lots of people have this issue and have had for a long time.

- https://www.reddit.com/r/Tailscale/comments/pm49t9/tailscale...

- https://forum.tailscale.com/t/high-battery-usage-in-ios/1152

- https://github.com/tailscale/tailscale/issues/3363

Weird, 4% on my iPhone. It's always on, though I don't make a lot of connections through it

You need to look at system usage because it’s the tunneling that’s draining the battery, afaict

I had this issue as well, it turned out that something was multicasting frequently, which gets turned into a broadcast because of wireguard's nature. I just switched to plain wireguard with a VPS and my issues magically disappeared.

You can use netmaker and the native wireguard app

I love tailscale, too. Also, I read this article before it had any upvotes and learned absolutely nothing new or insightful. Wish the author had kept going.

It was news to me that Tailscale allowed DNS lookups for particular domains to hit specific resolvers and those resolvers would serve up the internal VPC address, so you need nothing except a subnet router inside the VPC to be working against your secured AWS resources.

It means that you can close all the open ports on your VPC security groups without changing the configuration of how your external systems access the internal AWS services.

It was probably obvious to everybody else, but after I worked that out, Tailscale became my network.

Magic DNS is a fucking nightmare behind the scenes, I just use raw dog IPs to KISS. The TS app makes it easy to copy to clipboard.

See: https://lobste.rs/s/v4obi8/how_does_tailscale_s_magicdns_wor...

I also set custom hosts file entries for non-mobile devices and kablam, no magic needed (for personal use, anyhow).

Behind the scenes they’ve obviously got a lot of crazy to deal with, but it seems to work well from the outside (just using it for Tailscale lookups, at least).

Have you had issues with it otherwise?

EDIT actually I do have one gotcha. There’s a switch in the admin panel to override real dns. In theory if you changed that option and your machines were currently using private dns on route53 to find each other you might be in trouble (don’t ask me how I know).

There is no open-source equivalent implementation available for the PITA that is Magic DNS, so I don't use it. This ensures that if Tailscale goes rogue, I can replace it as easily as possible.

I thought headscale added support for MagicDNS? There's at least one other comment in the thread mentioning this. It didn't always have that support, but it has been added and the gap narrowed.

I’ve just had a look and GitHub and that appears to be the case.

Note, I know nothing about Headscale; I’m happy to pay for Tailscale and support the team behind it.

And I'm just as happy to use it for free, I haven't set up Headscale either as attractive as it sounds to own my own infrastructure when I can have someone else do it for me...

Tailscale is a really great service and it's so easy to teach someone else how to use it, compared to like every other VPN ever!

AFAIK, you could register your nodes in Consul and then use its DNS interface to resolve them.

It’s not DNS

There’s no way it’s DNS

It was DNS

The only thing worst than “It was DNS”, is “It was DNS, but in this rare and weird edge case only so it never showed up when you tried to debug it”…

I mean, I’m impressed by that capability. But I’m horrified by the potential future support implications. Who’d want to be debugging a problem with “magic DNS” at 2am on a Sunday morning while Prod is down and the entire C suite is half drunk, tired and angry, and breathing down your neck?

Sadly almost none of that is complex or surprising if you’re used to dealing with DNS deployments on Linux or BSD. What’s new is bundling a custom system name resolver for machines that can’t forward matching domain requests to specific name servers. Users are often left to their own solutions if they don’t use systemd-resolved or NSS.

Quote: "In the "before tailscale" times, if I needed to test against the production AWS resources or connect dBeaver for database maintenance, I would edit the security group to add my IP address, do my testing, edit the security group to remove myself. This is as error prone as it sounds. I quite often forgot to remove my IP address from the allowed addresses, a major potential security risk when you are travelling."

My takeaway from this is that the author was either lazy or lacked the knowledge to create an automation script that could've done that automatically (the add/remove) based on location. If that's the whole reason for this tailscale praise, kinda of takes away the tailscail actual usefulness and why it exists in first place.

Author here. Yep, I'm lazy and a cheapskate, so using the built-in AWS solution I thought was too expensive for something I only did occasionally. 99% of the time I'm hooked up to my test system rather than production (as it should be).

To add to the OP’s article, Tailscale can map IPv4 to IPv6 addresses when using subnet routers.

Imo this is incredibly handy, as if I want to expose a device to my Tailscale network, I don’t want to have to think about finding an IP address range that won’t conflict with the various local network ranges that my Tailscale devices are on. Especially if you’re using Tailscale in various corporate environments where 10.0.0.0/8 is used a lot.

Now I can just expose e.g 192.168.122.0/24 to my Tailscale network but it’s exposed as a unique IPv6 /120 prefix.

https://tailscale.com/kb/1201/4via6-subnets/

How about the old solution of devices connecting to an access VPN running on a nearby AWS VPS?

I have my private network right now. As a plus, devices can make direct connection when they are in restrictive corporate networks (allowing only 443/tcp). Less third parties involved. Seems more secure for personal use.

Sure, it’s not a mesh network, but that doesn’t matter if VPS and devices are in the same region.

But I get that mesh VPN products can be valuable to small businesses: ease of use, ACLs, SSO, central management.

There is nothing wrong with a direct VPN, but Tailscale makes them look hard to configure and you don't get the magical DNS stuff they provide.

Custom peer to peer networks over Wireguard are theoretically more secure. A list of additional potential adversaries who can remotely access your Tailscale network without client manipulation:

- Tailscale’s coordination server

- Tailscale’s cloud provider

- anyone who compromises the coordination server

- SSO provider you have chosen

- SSO’s cloud provider

- anyone who compromises the SSO provider

I consider myself relatively technical as a software engineer, but networking isn't my forte. I still don't understand the documentation about bringing in my other devices that don't have clients. I'd like to be able to have access to homekit while out and about on 5g for example - via tailscale.

Set up a tailscale subnet router on your home network, it can share access to clients not running tailscale. I've used that to access a bluetooth router that I didn't install tailscale on.

Yes I'm aware [1]. I would not describe these steps as user-friendly.

[1] https://tailscale.com/kb/1019/subnets/

Those instructions are a bit over-egged.

All you need is

  sudo tailscale up --advertise-routes=10.0.0.0/24,10.0.1.0/24

Then you can get to anything via it's internal IP address. I have a bluetooth router on my network that I don't want to install Tailscale on. It's on a permanent address of 192.168.1.13

I do this on the router inside my office subnet:

  sudo tailscale up --advertise-routes=192.168.1.0/24

I do this on my home debian machine (lets say it's on 10.1.1.13):

  sudo tailscale up --accept-routes

From my home network I can now open the web management page on 192.168.1.13. You can go further with DNS stuff but you don't have to.

But since it’s user space aren’t you leaving a ton of performance on the table?

This is one of the sell ng points of netmaker, besides three fact that it's all open source.

Very cool! I’ve found my new weekend project. Thank you!

I really love tailscale for my private network.

Nevertheless, I still hope they will revisit support for proper linux kernel wireguard sometime in the near future. This would allow to ditch separate meshing technologies for connecting server nodes (and routing into separate subnets via a tailscale subrouter node). Best of both worlds - ease of use and performance.

I really only started to have a look at other tools (like netmaker, netbird, innernet, wesher) because of these performance caveats (wireguard-go).

https://github.com/tailscale/tailscale/issues/426

netbird is a good alternative that has significantly better NAT punching (at least on my network), going through 2-3 layers of NAT just fine. The problem with it is although it does support kernel WireGuard, it works only if both nodes are on the same subnet, or if one of them has a public IP (port forwarding doesn't work, it needs a public IP). Otherwise, it creates a tunnel through NAT (using WebRTC's ICE) and then routes data from the WireGuard interface through that tunnel. This involves copying data multiple times (application → kernel → wireguard → netbird client (userspace) → kernel → network).

So unless you're fine with that limitation, look elsewhere for now.

I really tried to like netbird, but their current hard dependency on auth0 (also when selfhosting) unfortuantly put me off. Your statements regarding NAT punching are on the other hand very interesting.

I'm currently trying out innernet, mostly for the interconnected server nodes (k8s, not for actual real users). Seems to work fine (double NAT + single public coordination server).

Unrelated to Wireguard: I really liked Nebulas certificate-based client setup and its reduntant lighthouses (public coordination servers). May be an alternative if you want something at least a little faster than wireguard-go based implementations.

What is tailscale doing, exactly? They bill themselves as a VPN, but honestly I'm really a newb with networking (tend to be more on the data engineer/science side).

Tailscale provides a virtual private network (VPN) - in the old school definition - the network is private to your own devices/network.

You use it to connect your home network for example, on your laptop when you are out and about.

Any internet routing is purely done by your own isp provider (if you chose to tunnel internet traffic through tailscale)

Behind the scenes, it uses WireGuard which provides the vpn tunnel to your own network.

Tailscale also provides an authentication layer (ie username/login/passwords)

Thanks!

Wireguard isn't a chatty protocol, and uses very minimal. You can use it on a spotty connection and your internet will go through that.

> Tailscale also provides an authentication layer (ie username/login/passwords)

Unless I'm misunderstanding you, I don't think it does (having to use a a third party to sign in is one of the main reasons I haven't tried it yet). From https://tailscale.com/kb/1013/sso-providers/: "Tailscale never handles authentication itself."

Tailscale seems to dominate the hn front page. I've written an ask hn about that and I'd like some honest opinions:

https://news.ycombinator.com/item?id=31957059

Have setup tailscale and pretty happy with it, yesterday enabled TLS so my host have https pretty easy with caddy and reverse oroxy, problem is you can't use subdomains, so every host can only forward one https connection. My main server host several selfhosted services and want them all over https. Have googled an question open at caddy and tailscale forum, anyone here know what route to take?

I use Tailscale as well for my personal stuff (services in various clouds, remote gateway when I'm away from home, etc.) but as someone who defines best practices for enterprise cloud deployments I have a nagging worry about other people using it in work environments as a way to circumvent security guardrails.

That said, I love it and use it extensively from my iPad to work on a personal Gnome desktop via RDP.

Can someone comment on how safe/secure tailscale is? I am not talking about the wireguard part but the closed source part and the company.

> the closed source part

P2P client:

On iOS/windows/macos it doesn't make sense to complain about closed source code running on your machine; on linux/bsd/android the client is open source.

Control plane:

A compatible open source server is available & you can self host; the tailscale-hosted control plane is closed source and could, in theory, instruct your devices to connect to additional peers.

> and the company

The company has several high-profile employees with excellent reputations, which is typically a good sign of legitimacy.

> The company has several high-profile employees with excellent reputations, which is typically a good sign of legitimacy.

I’m sure people said that about Larry and Sergey back in the day…

People don't talk about Google not being a legitimate or unsafe company.

Issues with customer support? Sure. Issues with shutting down some products HN loves? Sure. Getting hacked or being insecure? No, in fact they probably have one of the leading security teams (Project Zero) in the industry.

Nothing lasts forever, but very little changes overnight either. If you're paying attention, you can watch the arc of growth and decay in many places.

Don't connect your GitHub account to it.

I made the mistake of doing this, it seemed to be working fine, and then all of a sudden to add a new node it needed access to my company's GitHub repositories in order to continue. (no way to continue without accepting)

So I could either give them access to sensitive company information, or remove my github account from my company, neither of which I wanted to do.

I just stopped using TailScale, I've been meaning to set it up again with a different login, but the experience left a bad taste in my mouth.

I my experience for purpose of home server which should be accessible for everyone, cloudflare tunnels are better, because tailscale solutions require every client to install their app to use the network. It is not convenient if you want to share some webserver with your friend and just send him a link.

True, but that is also not what Tailscale is for - tailscale would be me comparable to Cloudflare Access instead of their tunnels.

Tailscale has this in beta at the moment https://tailscale.com/kb/1084/sharing/

It does seem like a great solution but we've shied away from it because the price. For an enterprise you need the business license. Something like OpenVPN Cloud I think is 1/3 the price.

That's reasonable, and OpenVPN is ~1/10th the performance.

What's the difference between using tailscale vs the AWS VPN?

The AWS VPN, client and site to site is tedious to set up. Taildcale on the other hand couldn't be easier.

I’ve never used the AWS VPN, but the Tailscale VPN but the two things they’ve gotten right for me are:

- it has been unnoticeably reliable. We were early-ish adopters right as COVID hit; we’re a hardware company and have it on a handful of very remote devices/sporadic power and LTE devices as well as our development machines. The ARM binary _just worked_ on the embedded kit and I often remote in even while the payload is being flown. Automatically picks up right away when in network range, and I don’t ever wonder if I’m going to be able to connect to something

- it’s wonderfully simple to set up. Download the client, sign in using your org’s SSO (or personal email, but that’s for _only you_ and you can’t share with other users), and your machine just magically joins the network, gets assigned a fixed IP, and you’re on the network with everyone else.

yes to the unnoticeably reliable part.

during the covid shutdown I had to spend long periods of time away from my apartment (I decided to lock down with my family 3000km away from where I lived alone)

I had only a small window of time to set up some kind of tunnel back to my apt/set up some kind of remote access. I just installed tailscale on an old mac mini, connected a dinky spare usb webcam, and started photobooth just as I had to leave to catch one of the few flights that hadn't been cancelled due to lockdowns and restrictions.

It worked awesome. I could remote in, use the mac to access my backup hard drive, and watch the photobooth camera. The only thing I didn't forsee was the webcam was useless during the night cause I didn't leave a light on so photobooth would just show me a black screen.

During the day though, the camera showed me my worldly possessions and helped me monitor my apartment so in case anything happened I could call the superintendant of my apartment. It worked really well, and worked for 3 months until the connection died. I only had 1 month left so I left it alone.

4 months after leaving my apartment, I went back and found out the mac mini suffered from hardware failure (would no longer power on), so the tailscale vpn worked and was more reliable/more solid than the hardware it was running on.

After that experience, I am sold on tailscale. They're awesome.

I use both tailscale and cloudflare access and found tailscale is easier to deploy however cloudflare access also has tunneling which is pretty useful.

Do you not consider a VPN a tunnel?

I mean Cloudflare's public facing tunnel solution. Of course VPNs are also a form of tunneling, but as far as I am aware tailscale doesn't have a public facing solution.

Why does this read like a thinly veiled native ad ?

So do most of the comments!

I wish I had Tailscale when I was playing LAN games in college. Spent a lot of time debugging around network issues.

I hadn’t even considered this, this can be used as a modern replacement for hamachi in this particular scenario!

why not just use ssh tunnels with compression ON when you connect to RDS?

is tailscale like forticlient vpn ? so I can ssh into my instance that is not exposed into the world? so my instances can talk to each other without going through the public internet?

Tailscale can punch through NAT but requires outgoing internet access

Reply on news.ycombinator.com