https://www.vaultproject.org https://github.com/hashicorp/vault
It comes with both, a full blown PKI (want a new cert? Use an authenticated REST endpoint!) and SSH backend. On top of that you can use it to manage accounts for many other third party applications as well (e.g. PostgreSQL, MySQL) while leveraging a multitude of authentication backends to delegate granular access for all of these features.
It's been an absolutely eye-opener for somebody like me who's used to managing SSH keys/PKIs (what a pain) and I wouldn't want to use anything else right now.
This model is described in an excellent post by Facebook from a while back [1].
(Disclaimer: I used to work at HashiCorp, and put this model into production there, though the Vault support for issuing short-lived certificates was added after I left)
[1]: https://code.facebook.com/posts/365787980419535/scalable-and...
It's not a difficult problem, mind you, but there was custom code written that runs on developer laptops (OS X and Ubuntu) to support this workflow.
(Despite being a very similar looking string of bytes as more traditional pub/private keys, it's different in the SSH-Agent protocol, so don't assume all ssh-agent-looking daemons support it.)
That link doesn't work. The correct one is https://www.vaultproject.io
It creates and removes local accounts and manages the ssh keys and sudo permissions centrally, so you don't have to worry about not being able to get in if your LDAP/AD is down. (Our Enterprise edition, self-hosted in your VPC or in your DC, can optionally integrate with LDAP or AD for dashboard logins, MFA, etc.) We also have an AWS server edition in the AWS marketplace. When you remove someone in the dashboard, their account is removed across all of the servers they have access to, and their sessions are terminated, but their home directory is retained for archival or forensics.
Our primary foci are paranoid levels of security (hardened, all data encrypted at rest with X25519, in motion with TLS), flexibility, and speed: fast to deploy or integrate (w/ ansible, chef, puppet, terraform, cloudformation), and fast to install (milliseconds), with both SaaS/Cloud and self-hosted versions and open source python agent (we call it the 'shim').
We were founded in 2011 and serve thousands of servers and users in real time; I'm first.last at userify for any questions, and/or info at userify.
> paranoid levels of security
> # paste this into your terminal
> # and get started in seconds..
> curl -# https://usrfy.io/signup | sudo -sE
:OIn order to be effective, signatures have to come from a root of trust. Where are you forming a trust basis for a signature in the curl? The GPG sigs built into your package manager are signed by the packager and form a cascading tree of trust, and the TLS certifcate authority system is also supposed to form a foundation for trust. (Sigs are on the file, but where do you get the packager's first signature in this place? curl from https piped into sudo doesn't decrease the CA trust model and a sig on the file as well doesn't add anything, since the sig would be easily replaced by anyone with the wherewithal to mod the original file.
Instead, that sig would be security theater, much like an EV-TLS cert. (Not that security theater can't still be valuable from a marketing perspective from people who think that checking sigs on an https site is still valuable.)
While there can be security value in sigs, it doesn't come into play in these circumstances without starting from a position of a signed sig from somewhere, and ultimately the smart and paranoid will still curl to a file and actually read the script. Actually, ultimately, if you are in a security sensitive situation, you are probably not in the cloud at all, or on dedicated instances, and then you should avoid SaaS and cloud software altogether and look at an on-premise solution like Userify Express/AWS/Enterprise, as then you will have that crucial foundation of trust in your initial purchase and you can tightly control that environment.
Are you actually removing the unix account? How do you manage uids? How do you prevent reuse? How about over NFS?
Here's the source code: https://github.com/userify/shim/blob/master/shim.py#L161
If the software doesn't manage UIDs internally then this is a recipe for disaster because keeping UIDs in sync is a PITA. It's a waste of time to do manually, but an even bigger waste of time to have to fight with the OS to get it right. If you're using NFS then you're screwed.
in my homelab I'm running RedHat IdM (which is their downstream version of _freeipa_). It's some value-add on top of LDAP on the server side, and sssd on the client side. My IdM runs in a VM on a server that isn't always powered on, and I'm still able to login thanks to sssd being configured to cache.. something. Clearly I haven't played with it as much as I should :).
Users can log onto machines if the credentials are cached, I think. Unsure how PAM handles this on Linux.
The hotness these days is `sssd`, which transparently tries multiple directory servers (be they AD, IPA, or straight LDAP).
* Ensure that the following key is present.
* Ensure that only the following keys are present.
You can accomplish the second one with the 'exclusive' flag.
That said, I do like the "exclusive" tag too, I just tend to have a few one-off keys laying around as technical debt.
https://gist.github.com/tristanfisher/e5a306144a637dc739e7
Having a partly encrypted YAML file is also available since version 2.3.0
Key management and deployment are two different things. Installing up people's private keys on the hosts, doesn't involve any management. It's a standard sysadmin procedure.
Key managements involves policies such as: key rotation, algorithms to use, access restriction control, etc. It is way more daunting and complex. I don't see how ansible what ansible has to do with this, vaultproject on the other hand has most of the these features (and more) build-in. Never used it, but I know about it through a podcast presentation of vault in "Arrested Devops" IIRC.
I'm using it to migrate us over from keeping our SSL cert in Dropbox and 1Password and it works well.
SSL: AWS KMS style solution which predates it on internal, and new system built on KMS. These systems are merging as KMS takes a lot of the load off. Then it's down to building a key distribution system. All secrets actually stored in this type of system. Devs rarely access secrets directly. Lots of nice "client" wrappers which look like DynamoDBClient, or Mysql driver but actually fetch passwords on a 5 minute rotation from the secret store so a rotation means push new key, wait 5 minutes, pull old keys. Secrets preferably never hit disk.
Not a big fan of vault because it wants to connect in to your hosts to manage and rotate passwords.
This is a new service we recently launched, so any feedback would be greatly appreciated! My email is in my profile, if you would like to reach me directly.
[1] https://gitwarden.com [2] https://gitwarden.com/blog/2017/05/04/first-steps-with-gitwa...
Aren has been awesome with responding to emails and helping us set it up too.
This means you can set up a linux group with sudo capabilities (sudo or wheel, usually) in /etc/sudoers. Then using Foxpass you can manage the membership of that group by adding users on a permanent or temporary basis.
I'm a fan of public key distribution via directory service (LDAP) with failover/caching via sssd. Private key storage on a smartcard (Yubikey in my case)
SSL: It depends on where it's being deployed -- AWS apps use ACM, situations where I need to access the secret from within a running system image (gce/ec2) generally uses the platform's kms long with their object store.
Can't help but notice this account only has two posts, both asking about commercial solutions folks use for secret management. I'm gonna take a stab in the dark and say this is market research?
I don't even want anyone (including me) to have access to our SSL certificates. There's no lock-in, I can always change my mind and buy new SSL certs somewhere else.
This system uses a set of untrusted nodes that form a permissioned blockchain. Updating the chain requires a threshold of keys stored in the first block. The private keys are distributed over laptops/phones.
A person can have multiple devices that accept/deny new keys, while the servers check periodically for updates and can verify the new ssh-keys are legit by verifying the signatures.
I did a small demo at HotPETs 2016: https://www.securityweek2016.tu-darmstadt.de/fileadmin/user_...
I also hope to have it running again, soon. If anybody is interested, don't hesitate to contact us at linus.gasser@epfl.ch
https://medium.com/netflix-techblog/introducing-lemur-ceae88...
The private key never leaves your phone. Pair your phone with your computer to create a secure channel (channel is encrypted + signed with session keys only known to your computer and phone). Every time you SSH, the computer calls out to your phone over this channel and asks you to approve a signature.
SSH logins with simple push notification approvals. Code is public: https://github.com/kryptco.
Updates to either would be passed on to Puppet and distributed automatically across 1400 machines.
(Edit for typo)
Everyone puts their hsm keys on their github account (and removes all others).
We fetch the keys for each user from github on system init. e.g.
When we need to add/remove people, we just update the list of usernames in the script that fetches keys, and then kill off instances one at a time to force a redeploy.
How do you ensure that nobody adds another key? I don't think github gives organizations visibility into key changes in user accounts.
>https://github.com/sneak.keys
How do you do that? I mistook you for a Github employee at first.
Depends on the type and number of bits in the key.
If you create an ssh key that is already broken (say you managed to generate a... 512 byte RSA key), then an attacker would know what key he needs to generate before he attempts to authenticate with your server (or github).
But in practice public keys are meant to be public... very public. Like GPG keys! Here's a debian signing key https://ftp-master.debian.org/keys/archive-key-7.0.asc .
We can even verify that it's an RSA key with 4096... exactly what you could (should?) use to generate SSH keys. Effectively posting your public ssh key in the wild is as safe as debian posting their public signing key :)
``` pub rsa4096/0x8B48AD6246925553 2012-04-27 [SC] [expires: 2020-04-25] Key fingerprint = A1BD 8E9D 78F7 FE5C 3E65 D8AF 8B48 AD62 4692 5553 uid [ unknown] Debian Archive Automatic Signing Key (7.0/wheezy) <ftpmaster@debian.org> sub rsa4096/0x85215E51ADD6B7E2 2012-04-27 [E] [revoked: 2014-03-17] ```
I dislike that Github doesn't explicitly mention that it publishes your public keys, because they can be used to figure out your identity across multiple services. I believe someone a while back posted a demo on HN, where you could SSH in and it would greet you with "hello $yourname", which it derived from your github keys.
My advice: if you use different (user)names for different services, you should probably consider using different SSH keys for them as well. That is, if you don't want those two identities to be tied together.
The main problem that Fhilippo pointed out with the service is that ssh by default gives all keys present inn your keyring, or all keys named id_{rsa,dsa,ecdsa}. What should be done is to never present ALL your keys, but turn on "IdentitiesOnly yes" in your SSH config.
At least github supports two factor auth, but you're still placing trust in a third party.
It supports caching to etcd (incase GitHub API is unavailable).
It's specifically being built for particular systems to consume, and slowly generalised as it goes. The first consumer is BOSH. Typical BOSH deployments can have dozens or even hundreds of sensitive credentials to manage. Across a large company it quickly escalates into the thousands.
edit: typo
As much as I hate do admit it, but I belong to the small Linux minority in my otherwise Windows only business.
We only have ~10 employees. About half of them are technical staff with some access to certain systems (git, mostly). Only two users have global root access.
Vault (which was mentioned somewhere in these comments) looks interesting, but at our current scale it looks like it might be overkill.
chef and hashicorp vault
Another neat thing to deploy into dns is sshfp records so there's almost never ssh fingerprint verification prompts for deployed hosts. Alternatively, ssh host fingerprints can be deployed to LDAP.
For those wondering, [1] provides a bit of a background on SSHFP records. You can only skip host-key checking entirely if it's served with DNSSEC, although that might be easier if you're running internal DNS.
How do you have your system working? Its something I've fiddled with briefly, but ultimately gave up on for now.
SSL Certs - vault PKI (internal) AWS ACM (external)
These two issues are known. The next step is notifying users via e-mail of expiring certs. Then the account button will be functional.
We're validating the value proposition. I've a real need for this application, but we're trying to see if others have the same need.
Would you like to be notified when the account and reminder functionality goes live?
That said for these services it works very well, except there are some issues with it lagging on sending confirmation emails that just make you retry.
no idea how secure this is or what implications this could have but it's easy and works well for our use case.
shadowc - client of shadowd