Infrastructure Security Without Slowing Down Developers | Hacker News Reader

Infrastructure Security Without Slowing Down Developers

gravitational.com

37 pointsold-gregg15 comments

For the author there is a little typo in the article. It says "This is problematic for three reasons", while listing two reasons.

The last section talks about choosing proxies over jumphosts, without mentioning what would be necessary to setup such a proxy. I would be really interested in a more indepth explanation of how such a proxy would work.

The author compares the proxy to a "dumb switch". How do you authenticate users without terminating encryption? And, if you don't authenticate users on the bastion, why bother with a bastion at all?

The proxy does not authenticate users. When a user shows up without a certificate, a proxy simply redirects to an OAuth2 / SAML endpoint such as Github or Google. When OAuth redirect comes back, the proxy forwards user's identity to the certificate authority (usual SSO flow), which issues an SSH certificate which gets sent back to the client.

Now, "armed" with a certificate, the client can try again, in which case the proxy simply forwards the request to the destination host, which verifies the certificate. It is the same as having a non-terminating HTTP/TLS proxy/load balancer, just for SSH.

Ah, interesting! Thanks for your response. What do you use to get this work flow going? Would be really interested in the underlying infra. Do you have any pointers to get me deeper in this subject?

The entire thing is on Github: https://github.com/gravitational/teleport

Quickstart: https://gravitational.com/teleport/docs/quickstart/

To quickly answer your question: if you rename teleport client "tsh" to "ssh", trying to use it without a certificate will trigger SSO workflow. The alternative is to issue "tsh login" CLI command in the morning.

I really like the certificates for SSH idea... I've never thought about this and have always used vanilla keys.

Does anyone have experience using this? Besides giving auditing information, does this have any other benefits?

Author here. I mentioned a couple of other benefits in the post, namely TTL (time to live) and the ability to have fine-grained role-based access control. Your servers should use certificate-based auth too, to join a production environment. Then you won't have the trust-on-first-use issue, because server authentication simply compares the CA of a client & server and you don't have the dreadful "are you sure [y/N]" question when connecting to a box for the first time.

Basically, certificates rule the world everywhere, but due to rough UX, they haven't seen broad adoption for SSH. That's until now, hopefully :)

[1] https://en.wikipedia.org/wiki/Trust_on_first_use

Thanks for the follow-up! I've always thought the TOFU thing was a huge issue: it's hard to verify the first time and rotating server certs is a nightmare.

Are there any products/tools that make this easier? I've noticed that this isn't common even within companies with robust security infrastructure.

> Are there any products/tools that make this easier

You just read an article by Gravitational, authors of Teleport. Check them out.

The opensource product is pretty full featured, the commercial version has a few features companies usually need (I believe SSO is one of them). Now instead of having people submit a ticket, send the keys, yada yada, you can just point someone to the portal and they can just use teleport credentials (or if you have the feature, the corp credentials). And the session is recorded. You can also use the command line – and it still gets recorded.

Disclaimer: We use Teleport in our company. I pitched after finding out about them here in HN. We still have bastion hosts, which now automation automatically deploys teleport and they show up in a single place.

My own solution to this is to store the server key fingerprints in a public Git repository, which can then be pulled from on all devices that need to be able to connect.

This works nicely for my own personal setup with a few servers. It may be possible to scale this to a larger environment with multiple users too.

Then main benefit is that whenever a server is rebuilt (which can be quite regularly when using infra-as-code/infra automation), I don't have to go and manually update every single client that needs to be able to connect.

https://www.jamieweb.net/blog/managing-your-ssh-known_hosts-...

I'm one of the maintainers of https://github.com/gravitational/teleport

Another one to look at is https://github.com/Netflix/bless

Also, if you want to stick with what you have (most likely OpenSSH), we wrote an OpenSSH guide: https://gravitational.com/blog/how-to-ssh-properly/

Some netflix engineers have given some pretty good talks about SSH certs.

There are a few floating around, I think this is one of the good ones: https://www.youtube.com/watch?v=JwLGsWYVjqU

I like the jumphost model we’re using in our current AWS accounts: dynamically-generated ecs instances that automatically shut down after a certain period of time.

Heh, that's a neat idea and cheap to implement on top of existing SSH tooling!

Is that model powered by something open source? I'm interested in learning more

It’s just a (mostly) simple bash script a co-worker wrote.

It relies on Amazon’s Session Manager plugin, importing the user’s public key with aws ec2 import-key-pair, and firing up a new instance on the relevant subnet with the appropriate role, security group, etc.

I rather doubt the company would allow even something this simple to be distributed, unfortunately, at least without lawyers getting involved, hardly worth it for a 100 line bash script.

Reply on news.ycombinator.com