Teleport 2.0 Released: Modern SSH for clusters and teams | Hacker News Reader

Teleport 2.0 Released: Modern SSH for clusters and teams

gravitational.com

133 pointscraigkerstiens72 comments

I seem to remember that the features now available as Enterprise-only were not listed as such during development. Even if that's not the case, it's still a bummer. And it really sucks having to go through a salesperson's spiel just to get an idea of the costs. I hate it when companies don't post list prices on their site. I'm also looking at you, Hashicorp, and your Vault Enterprise.

>I hate it when companies don't post list prices on their site.

Same here. If SpaceX can openly advertise their prices, then so can you.

[0] http://www.spacex.com/about/capabilities

To be honest, it takes one email to get our pricing, and we mostly use it to stay in touch and understand your use-case better.

Thanks for feedback though, we will work on making it easier to get the pricing and explain the split between enterprise and OSS parts of teleport better.

Actually by not posting your price upfront, is actually wasting time for both you and the client. You see if I have a budget for a solution of 60USD and your solution starts at 6000USD I know to look elsewhere. But if you don't what will happen is you will collect my email to "stay in touch" about a product I have no use for, increasing my spam by 1. My email address in your mailing list is wasting my inbox space and your time. In fact your mailing list is probably what you use to give your sales person. But it's counter productive instead of having 100 potential customer who have accepted your pricing for your product, now you have 10000 people who may or may not be interested. So now your sales staff have to spend time chasing up cold leads instead of focusing on 100 customers whom they have a higher chance of closing. So you see by not posting the price you are causing harm to your own sales pipeline and one more person to make you as a spammer.

Chances are if you need to contact the company for pricing details their product is not going to be in the sub-$100 range (or possibly even the sub-$1000 range)

So just to chime in on this, I'd echo others in saying that "call for pricing" or any variants thereof, are a huge turnoff to me in terms of buying software.

It instantly makes me think "this will be expensive" and "if I call and it's not right for me, I'll be getting hassled by sales people forever more"

Now, neither of these things may be true for your software, but people who've experienced the pain of buying "enterprise" software before will likely be put off by this approach.

It also gives the impression that pricing isn't fixed and that the sales process will involve lots of haggling, which also puts people who don't like haggling off.

And to be clear: it is your product and it's your choice how to develop it or make it available on the market. The only reason that I took the time to vent is that I actually like Teleport a lot. You won't see me complain about Oracle or Symantec pricing details. :-)

"Contact us for pricing" means "unless price is no object, forget about it" in my experience. Every time I've followed up with a "contact for pricing" product, the reply has been for 10-100x what I was expecting.

If you're selling to only Fortune 100s, that's fine. If you're aiming a little lower than that, not giving a public price of some kind up front is driving away a lot of potential customers.

On the topic of enterprise features and specifically, Hashicorp Vault, I did ask Hashicorp sales if it was possible to trial Vault Enterprise (for the UI) without the pre-sales dance and was flatly told "no".

This kind of attitude really rubs me the wrong way. On follow up from their sales I simply told them I opted for an open source UI from Github.

While I want to support the work Hashicorp are doing, I want to be treated as someone who understands that I don't need to spend money on the product. It's a concern to me that as these open source projects find ways to monetise they start to slowly make their way back to the type of ecosystem that I wanted to avoid in the first place.

Here is a question for teleport developers:

Why teleport over kerberos + ssh? What can this provide that RHEL7 + a kerberized LDAP like IPA + sssd already provides?

Well, with enough in-house expertise everything is possible using a set of the tools you're an expert at.

But with ever-growing complexity and volume of the infrastructure, the expertise required and time to manage these things are in short supply. Teleport is mainly the opinionated and simplified way to deliver SSH best practices.

Additionally, it has a few neat usability features (like a nice Web UI) and directly addresses the use case of managing the infrastructure that's not yours (MSP use case) where you can have multiple clusters owned by different organizations where you can set up cross-cluster trust via certificate authorities (CA)s [1], well... that's just what's on top of my head.

There's more here: http://gravitational.com/teleport/ or feel free to reach out and chat, we're a friendly bunch! :)

[1] That's our use case, Teleport was originally built to set up and manage Kubernetes clusters on infrastructure located behind firewalls, see our flagship here: http://gravitational.com/telekube/

Ah the MSP use case is what I missed, that makes a ton of sense. Perfect.

"But with ever-growing complexity and volume of the infrastructure, the expertise required and time to manage these things are in short supply. Teleport is mainly the opinionated and simplified way to deliver SSH best practices."

We're talking about the ssh command here, right ? And associated key generation and config files ? Are there people who are "experts" at that ? Like, the same way there are people who are "experts" at can openers and chip clips ?

"Additionally, it has a few neat usability features (like a nice Web UI) and directly addresses the use case of managing the infrastructure that's not yours (MSP use case) where you can have multiple clusters owned by different organizations where you can set up cross-cluster trust via certificate authorities (CA)s [1], well... that's just what's on top of my head."

You just wrote slide #3 in a future blackhat presentation.

Kerberos + ssh is amazing. I don't know why more people aren't aware of this holy grail setup.

there are a couple of nice SSH setups you can achieve with RADIUS and our friends at https://www.foxpass.com/

Vault also has SSH backends that you could find useful

My thoughts exactly, and it has years of production battle hardening + crypto validation.

Kerberos is really complicated. I am currently setting up LDAP, and have it storing SSH keys. Which is a far simpler setup.

We do both. Ssh keys are the backup if there's ever a kerberos outage.

Also make sure to deploy the ssh keys somewhere the users can't write for extra security. Don't allow them to control their own authorized_keys files! :-)

Hey HN, we (gravitational) are pretty excited about this release. We'll be monitoring the thread in case you have any questions.

Hi twakefield, congrats on the release!

So does this seem like a typical use in an AWS VPC:

- SSH bastion server with public IP with security groups for each user

- Every other server only has private IPs, only allowing access between each other and then the bastion server

How do you generate and store keys? Are you using something like Hashicorp's Vault?

Hi atonse, Russell from Gravitational here.

As far as configuring your VPC having the bastion (Proxy) as the only server with a public address is reasonable. One of the nice things about Teleport is that the Teleport Proxy itself doesn't have access to much, so exposing it to the Internet is fine. The Auth Server is the one that holds sensitive information and we recommend you create a security group for it and only allow it to be accessed from Teleport Proxies or Teleport Nodes.

With respect to keys, they are stored and accessed via the Auth Server in Teleport. We recommend you have strong access controls on the Auth Server. If you are using the default backend (BoltDB) or directory based backend that's all you need to do. If you are using etcd we recommend you have strong access controls on the server that runs etcd as well as etcd itself, we have an example in our Teleport repo for etcd configuration if you're interested[1]. If you are using DynamoDB, we recommend having a strong IAM policy. We are not using Vault at the moment.

[1] https://github.com/gravitational/teleport/tree/master/exampl...

Speaking of Vault, AFAIK the community expressed a lot of interest in providing custom secret storage plugins, so the latest releases have had a much simpler storage interface to implement, which is how DynamoDB was added, someone just sent a PR:

https://github.com/gravitational/teleport/tree/master/lib/ba...

Great, thanks for the info!

> Teleport Proxy itself doesn't have access to much, so exposing it to the Internet is fine.

yeah this is a bad idea in general. If you have critical stuff you need to SSH into from the public internet, keep it all in private IP space and have an openvpn gateway (or IPSEC VPN) with a public interface, and a private interface facing inwards towards the hosts.

you should not even be able to route to the IP of the thing you want to SSH to unless you've authenticated to the VPN and your client device has been handed out an IP in your RFC1918 IP space.

a machine like an openvpn gateway can also serve the purpose of getting you access into an OOB network (example: a public facing IP on a 100Mbps DIA circuit you've bought from a totally diverse ISP in the same colo, with a static /30), which has access into internal IP space devices such as serial console servers and ssh bastion hosts.

authenticate the clients by a unique public/private key pair per client device. Easy to revoke a specific device's key from the server side if needed.

> yeah this is a bad idea in general. If you have critical stuff you need to SSH into from the public internet, keep it all in private IP space and have an openvpn gateway

This is not a bad idea in general. In fact, teleport proxy implements this exact model you have just described, where only proxy is available and acts as a jump host to the set of machines available only on the private net. The only difference is that instead of open VPN gateway it uses SSH jumphost model.

Teleport proxy uses OpenSSH cert auth, in addition to that teleport node also does cert auth.

Not everyone needs to always set up VPN, sometimes jumphosts + cert auth are perfectly fine.

I encourage all of my competition to allow access to relay to critical internal things with only SSH based authentication on a bastion/proxy, and nothing else.

This is not how SSH jumphosts and Teleport Proxy work. With jumphost ssh client goes through the authentication and authorization twice:

* first when connecting to the jumphost public ip and requesting to execute a subsystem to allow proxying to some internal IP/host

Jumphost does not terminate the SSH and in fact it is MITM capabilities are very limited.

* Second time authentication and authorization happens when ssh client connects to the target SSH node.

This pattern is in fact quite modern and is being expanded in the beyond corp architecture.

https://research.google.com/pubs/pub43231.html

It deprecates perimeter security model that you mention via VPN gateways and replaces it with on-demand end to end access via controlling gateway.

Yeah but the security model it implements is vulnerable to any new remote code execution that pops up for OpenSSH.

With the VPN as your boundary, you can configure it so the VPN doesn't even respond to packets unless they are TLS authenticated. A huge security win, you're not exposed to random attempts to do remote code execution against your open sockets!

Critical infrastructure demands solid, understandable security (ie defense-in-depend). Throwing around shiny, new, unprovens things to "give the emperor new clothes" increases risks for major security breaches.

Plus, this commercial thing doesn't do anything pam ldap and config mgmt can't do. Just reinventing the wheel yet again to charge people money for proprietary "solutions."

"yeah this is a bad idea in general. If you have critical stuff you need to SSH into from the public internet, keep it all in private IP space and have an openvpn gateway (or IPSEC VPN) with a public interface, and a private interface facing inwards towards the hosts."

That's a ton of complexity when you could just run knockd on public facing sshds and make them disappear that way.

It's extremely tight, simple code - consisting of a single binary - and it never crashes or hangs.

No, I am not suggesting that you get rid of all of your keys and passwords and rely only on the knock for your security. (I have to write that because response-comment-numero-uno will strawman that to death). Keep your keys and passphrases in place and add the knock.

Port knocking is just the best thing.

Port knocking, what is this, 2002? Security through obscurity is not any form of security at all. Properly implemented public / private key crypto is not rocket science anymore.

As predicted.

The idea is, in addition to the normal security measures you use with sshd you also hide the service with port knocking.

Nobody anywhere, at any time, has ever suggested using port knocking as the sole means of securing your sshd.

Port knocking and some even obscurity are valid additional layers of defense-in-depth if combined fundamentals of A3E.

State actors can afford millions to spend on build/buying sploits for [insert technology]. For example, use different standard for OS at edges where possible to reduce attack surface. Preferably scrub network traffic at edges (not just web traffic) and really lock down traffic to remote access boxes.

Hi walrus01,

If exposing Teleport (or OpenSSH or any piece of software to be honest) to the internet is a significant concern, the solution I recommend is to put it behind spiped. Spiped has a very small and well written code base and well suited for this problem.

Because Teleport is fully interoperable with OpenSSH you just follow the instructions in the README to get it working: https://www.tarsnap.com/spiped.html

Yup. Open ports to the world is a terrible idea for anything real. This is why SSH jumpboxes (say a roundrobin pair of OpenBSD VMs with ssh rbash &| strongswan. Add secure portknocking to the pf firewall for bonus points.)

Plus, there's already plenty of ways to AAA OpenSSH using puppet/chef, PAM, RFC 4255, google authenticator (via pam plugin). It's really easy to set up if you've done it before.

Just to make sure I understand your reasoning right: VPN + SSH is better than just an SSH jump host because if someone finds a critical bug in OpenSSH, they still have to break the VPN, and vice-versa, someone breaking into your VPN still needs to also break SSH?

That, and the usage of unique per device public/private X.509 PKI key pairs per client, for connection to the VPN. A typical person might have 3 sets of keys:

a) laptop

b) home office desktop

c) android or ios device (phone/tablet)

then, of course, once connected to the vpn, to authenticate to ssh the person will probably be using their per person ssh public/private key pair from their workstation.

So we have the ability to revoke an individual client device's vpn keys separately. In event of total compromise we can revoke both vpn keys for device(s) and the person's ssh key.

Don't people normally have per-device SSH keys as well? I basically never copy them between devices... But I guess that could be harder to manage if you have tons of devices you might SSH into, especially if they are not all centrally managed servers.

What's Enterprise pricing look like for a small deployment? <10 systems

qmarchi - we have some different options that are dependent on support needed. If you fill out the contact form on gravitational.com I can go into more detail. Or email info@.

When I see "contact us" as a pricing model, I mentally note that this is likely a "we charge as much as we can, depending on who you are" model. I know that's probably not what you intend. If you could give some general parameters on price, it would help me avoid that association with companies that do this.

i'll save you both some time. "contact us" means at least $10k. if you don't want to spend $10k, they probably don't want you as a customer either.

This is a completely irrational thing to say (and is speaking for others) when you really have no idea what the facts of the matter are one way or the other.

Well, he is a cynic.

It's completely honest. When I tried to get an Elasticsearch license (when it was cheaper) sales refused to respond to my emails for a bit. Bugging someone on irc I finally got someone to respond and got a nice trial license but they /REFUSED/ to take my money.

I would have dropped the money in their pocket if they would have responded. But I wasn't asking for a license for a company but for myself... so they got stupid

It's not a truthful statement. If you, op and the others that voted me down want to believe it, there's not much anyone can do about it. All of you are irrational to believe that something someone says is the truth just because it rings true with an experience you had in the past, with another company, with other people running that company, with another product. What it is, is dogma, which is why I hate dogma.

to put even one SMART person's time onto a problem that costs less than $10k in aggregate is being an irresponsible, possibly stupid, business person.

the only possible exception may be a company with literally $0 revenue, i.e. looking for their first customer.

Maybe. Maybe not. So, it's irrational to believe that all desired outcomes may be obtained by limiting use of a resource based on its perceived value.

yeah, you wanted a license because you were going to hammer them with questions and suck up all their smart peoples' time, and then complain loudly when they didn't heed your beck and call for $5000, possibly in a public forum, negatively affecting their business to the tune of $millions thanks to google.

guess what: so does everyone else. also guess what: there isn't any money in that, because that time:revenue isn't built in to their business model, because they are a software company.

$5000 is a lot to an individual. when an individual spends $5000, they want to feel important. to a real company, $5000 isn't even a single paycheck for a single senior software engineer, let alone sales person.

the real reason for "contact us" first approach is to keep in touch with folks who are interested, understand their use cases better as we are in the relatively early stages of the enterprise offering

How about tell us the price?

I really dislike these kind of answers, and they really serve no purpose. I have a clear understanding about my potential budget for something like this. Don't waste my (and your) time by making me jump through hoops. I'll be sure to get in touch if I believe your product to be offering me value for money on my terms.

If you are not transparent on pricing, I start to wonder what the next guy is paying, and also, what else you are not transparent about.

Any CloudFormation template to launch this on AWS?

Hi nikolay, we don't have a CloudFormation template, but we do have a containerized version of Teleport that is fully configured with multiple clusters that you can use to test with.

https://github.com/gravitational/teleport/tree/master/docker

Did you know that you can do this:

  ssh -t user@machineB ssh user@machineA

That is, you can execute an ssh client command, over ssh. The '-t' option is required.

For the folks who are interested in such things: "SSH, The Secure Shell: The Definitive Guide" is an amazing book with lots of great info and nice tricks/SSH solutions

http://shop.oreilly.com/product/9780596008956.do

The -t creates a terminal on the remote host, so in essence it is the same as SSH'ing to machineB and then running the ssh command.

Even cooler is the use of SSH tunneling to set up a ProxyCommand.

To expand on this, the reason why it’s “even cooler” is because you can use keys to login (as opposed to password) without forwarding your ssh-agent (which is a security risk).

I wrote more about this technique at https://michael.stapelberg.de/Artikel/ssh-conditional-tunnel...

In a nutshell:

    Host home
        Hostname home.zekjur.net
        ProxyCommand ssh -4 dualstack -W %h:%p

Then, “ssh home” will log into “home.zekjur.net” via an IPv4 connection through “dualstack”. This is useful in environments where you don’t have IPv6 and your only use-case is SSH'ing home.

Why would you recommend this over -W?

ssh -W user@jumpHost user@destination

This is a very timely post for me, as I may have a very good use for Teleport and I hadn't heard about it before :)

have a look at jumpcloud

Is there an architecture overview for this case?

> Software vendors: they like Teleport for providing remote support of their products. Teleport can be used as a “remote control” to assist their customers with any issues of their software installed and running on-premise.

How does the customer control access to the app deployed on their premise?

Thanks!

Here's the overview of the feature we call "Trusted clusters"

http://gravitational.com/teleport/docs/2.0/admin-guide/#trus...

Thanks, this is perfect. Will try it out!

Has anyone else used Teleport in production? I would be curious to hear any feedback?

I tried it during the 1.x phase, and at rest the daemons would eat a core :/.

Hi Operyl, Russell from Gravitational here.

We've made improvements in resource utilization for Teleport 2.0 and hunting down any further resource utilization issues is definitely one of my focus areas for Teleport 2.x.

If you run into issues like this again please reach out and send us your teleport.yaml, we want to run these issues down and resolve them.

Just difficult to want to deploy it all again, I guess. Especially since you guys _still_ lack basic packages for common distributions.

That's unfortunate, thanks for the response.

Have you guys considered buying Fox Technologies? Do you have a compare/contrast between their ssh replacement and yours?

Anyone using it in production? How does it differ from other possible products/solutions?

This is awesome! Much interested to read audit results.

Reply on news.ycombinator.com