Ask HN: How are credentials managed at your company? | Hacker News Reader

Ask HN: How are credentials managed at your company?

138 pointsshubhamjain93 comments

Access to servers, SaaS apps, crucial infrastructure software. There are tons of things that need to be accessed by people in the organisation. Although, in many cases, there is an option to create new users but sometimes it's not and often, creating new users is more of a hassle every time someone needs access.

How does your company deal with giving and managing access?

View on HN

Our authority for authentication is Active Directory, it may sound very enterprise and not-hip but it's great because basically everything supports it. A user just needs one password for all our systems - email, servers, laptops, internal applications.

For servers we use LDAP, so you just SSH to a machine and it creates an account if one doesn't exist. It authenticates against AD every time, so if an account is disabled there, their access to all servers is instantly revoked. You can still use SSH keys so you don't need your password every time.

Internal web applications (mainly Rails) are backed by LDAP too, but also support NTLM SSO for Windows uses (so they only need to enter their password once when they log-in to their machine). Not everyone uses Windows (or is part of the domain), so we use CAS on top of that to make switching between applications transparent for everyone.

We don't really use any external services that we need to share passwords for.

We pretty much just authenticate as much as possible via our Active Directory. It just works, pretty much every self hosted service we have can authenticate via LDAP.

I assume that we could just run an OpenLDAP server or something, but Active Directory is the easiest way to get started with LDAP.

One idea I've had, but never got around to is a OpenID/OAuth service that could authenticate users via the AD.

Take a look at the open source project Keycloak (http://www.keycloak.org/), it does exactly that and takes very little time to setup. The easy way is pretty much:

  1. Run the docker container (https://hub.docker.com/r/jboss/keycloak/)
  2. Login to the web console
  3. Create a "Realm"
  4. Go to "User Federation" and add a "LDAP" provider
  5. Go to "Clients" and configure keycloak to handle OpenID/SAML for your app

Disclaimer: I work for Red Hat, and Keycloak is one of our projects. If you want support, we recently released it as RH SSO.

That's sort of what I wanted, but on steroids :-)

Seems a little overkill for my use case, but nice.

> One idea I've had, but never got around to is a OpenID/OAuth service that could authenticate users via the AD.

I've just implemented auth0 on a website that I run, solely because one of my sites/customers wanted to have the service I provide be available via single sign-on to their staff and their customers.

They store all of their staff in Active Directory, and their users in an SAP store.

auth0 gives me a simple OAuth API, and I just point to my customers' configured auth0 client, and they can configure their auth0 account to use Active Directory and SAP.

I actually liked the integration, wasn't hard to achieve and solved a hard problem.

Given that auth0 is free for up to 7k users signing in within a month, I may make this available to other customers.

I didn't think you got AD integration for free? I thought that was in their 'enterprise' plans?

That is in one of their paid plans.

However my customer needs vary, some of them will be good with just the auth0 free plans, some will choose the paid plans... some do not need it at all. The one who did need it found the price well within their estimate for it (they had a figure in mind for paying for the feature to be added which amounts to about 3+ years of auth0 service).

It's worth noting that it's safer to use Kerberos for the actual authentication part over LDAP and/or NTLM. In the event of the servers being hacked, you avoid leaking your password (but may leak some form of ticket, of course). Using LDAP to store the account details and authorization data (so you can create user accounts at first login) is fine though.

We just use "c0mp4nyn4m3" everywhere, unless we just use "companyname". Or the same passwords on multiple servers, or a few well-known passwords which are used again and again.

I've tried and failed to change this culture. Why have full-disk encryption if it's secured with companyname/companyname? Why encrypt backups if the password is 12345? Apparently, it's so we can show we've fulfilled our data-protection obligations.

I've seen this at every company, large and small, including a defence contractor, I've ever worked at. It's an unwinnable battle.

Second that. The nicest thing I've seen is at a large company, where you had to go through a very long and time consuming bureaucratic process in order to obtain the password (companyname123, identical for everyone).

I once issued an order to our service desk that qwerty123 was no longer to be given to users as a "temporary" password. They (the users) always said they would change it, but they never did.

At another workplace, I took Bruce Schneier's advice about writing down passwords (i.e, not a bad idea, so long as you keep the note safe) and gave credit-card sized, laminated cards to everyone with their unique and random password on it. Strict instructions to keep it in their wallet or purse. So these things turned up all over the place, stuck to desks, keyboards, and furniture next to desks. So much for that one. Unless people are getting fired for this kind of thing, they don't care.

I offered to crack my boss's email account to show how insecure our systems were, but he didn't act on it when I succeeded. But I did get the blame for hacking his email account a year after I left the company. Obviously, it was me (they reasoned) since "he's done it before!"

It's super useful though. If someone forget's the htaccess login credentials that are shared with a customer, and it's too embarrassing to ask if they remember, you can just run john-the-ripper on it, and seconds later: crisis averted.

twitches

Pretty much the same for me. At an early job I had at a very large software company (that did banking and hospital software), I was given developer access to a very old intranet application that they had (with the idea that I would tweak it if the need arose). All credentials were stored unencrypted in an Access database, and a little random testing showed that most people used the same password for the intranet as for their main computer account.

I've given up trying to change the culture at these places, and just make sure that I'm not part of the problem, and that my objection has been noted (for "I told you so" purposes, later on).

We use a simpler alternative as password - abc123, I mean what could go wrong. Our accounting and budgeting software was hacked 3 years ago due to it being setup incorrectly and accessible via google search. Since then our security teams have spent more time enforcing smaller rules (comparatively) like blocking flash plugins than asking business users to create stronger passwords. Any attempt to incorporate 2-factor or OTPs is met with "business users time is very precious they cannot be wasting time on using 2-factor or OTP."

They just won't learn, will they? If they've been compromised but still don't want to do the right thing ...

I feel like security is one of those things where no one cares about it, everyone procrastinates, and then companies invest a lot into it once something bad happens.

My primary role (these days) is as an Identity Architect. What that means is, I look at all the disparate systems in an organisation and design a better way to manage credentials.

My main client base is Blue Chip, so that means a majority of Windows based services, and a smattering of Linux, and even mainframe. Most companies have totally disconnected systems requiring manual user creation on multiple systems, with the user having to remember several sets of credentials just to do their daily job. This may be archaic compared to the proliferation of oAuth across the web, but large Corporates are rarely up to the curve, and in many cases are completely unaware the curve even exists. This is where I come in.

The main component of any credential management is an identity hub; I mostly work with Microsoft Identity Manager, but there are other options, such as IBM Tivoli Identity Manager and Cisco Identity Services Engine (to name but two). The identity hub takes data from [multiple] Authoritative Sources (HR, SAP etc.) and compiles a meta-identity of each user which is then forwarded as user attributes onto target systems. If the target system doesn’t need or care about ‘Job Title’ for example, that can be excluded from the forwarded data. Any user changes in the Authoritative Source are then replicated automatically to any target system on a regular schedule.

Once you have a Single Identity, it is then a case of leveraging SAML, oAuth, or plain old Kerberos/NTLM for login authentication. In an ideal scenario all systems will be able to support a compatible logon provider service but that isn’t always the case. Getting users down to a single id/password is always the end goal though.

Of course this also all scales out to the Cloud, with direct support for AWS, Azure etc.

If you want to know more, hit me up with some questions!

Do you deal with any situations where public/private keys are used instead of passwords?

You can store the keys in the Identity Hub, or in Active Directory - providing the logon provider is supported it should work.

I'm currently P2Ving a lot of old servers. Identity management is in my hit list for the year. So yeah, I'm interested and would like to subscribe to your newsletter

Cool - I've been toying with starting a small IDM blog for a while now, I might just actually sort it out now ... :)

I'm building a product in this space and would love to get your feedback if you're interested.

My email address info is in my profile.

Just a small note: Thanks.

Having joined a mega-corp in 2010, around 250,000 people, there was a big migration of 1000s of disparate systems to 'Single Signon', a slogan displayed under a login button. It was an ongoing process, and when that little slogan appeared under the login for nth intranet app, it was a small sigh of relief.

So thank you.

And an interesting description of how simple it seems to do given the other system plays nice. What do you do with things like SharePoint where users at any levels can add fields to profiles, and these fields can be changing all the time? Just have a massive profile template constantly adding fields?

This is where information governance comes in. It's fine to add bespoke profile fields in SharePoint (or any other app), and the Identity hub is will just ignore them if it's not got rule defined. If it's decided that the 'favourite animal' field is to be pushed out to other apps, then a new rule is added to the Identity sync process. It's all fail simple as you say; the main problem always tends to be departmental politics.

We use SmartCards which contain our private certificates.

The certificate on the SmartCard is encrypted by a passphrase.

The OS has been modified to support the SmartCard reader and use our individual certificates to authenticate and authorize us.

All our applications have been modified to authenticate and authorize us based on the decrypted SmartCard certificate and the roles we are in.

We can order roles through a centralized self-service web application. When a request is created, we get an automated e-mail with the request identifier, which acts as a file handle in the C programming language. The request then goes into the local security officer's queue, where it is either rejected or approved; if approved, it then moves into the role owner's approval queue. The outcome of the decision process is e-mailed to us automatically by the system. If approved, the access to the system or application in question is instantaneous.

Even SSH has been modified to use the SmartCards or soft token certificates for technical user accounts. It took us years working with Oracle to get their PKI fixed, since their in-house experts never saw a SmartCard reader, but eventually we got to the point where even the Oracle database uses the certificate on the SmartCard for authentication and authorization. Authorization from the web self-service application is translated into Oracle roles inside of the databases.

Even our source code management system uses SmartCards.

We run our own certificate authority. All of our relevant software is preloaded with the certificate authority's certificate by the respective engineering teams (component owners), so that the entire chain of trust can be verified. Our certificates use 4096-bit keys.

We do not use logins or passwords anywhere.

Sounds awesome. How do you use sudo? Just plain NOPASSWD and trust the SSH auth?

Sudo is in LDAP. We are allowed to run specific commands based on which roles we are in. Those roles are used to generate sudo roles.

Nobody, including 2nd level UNIX support, has or knows what the root password is.

If there is an emergency, the sysadmins can temporarily generate a root password, which is then automatically reset with random garbage after a few hours, so nobody will know what it is after that.

This is by far the most interesting scheme in this entire thread!

How do your software components authenticate to each other? With client certificates?

Yes.

Could the same thing or something similar be done with a yubikey?

This is still a hard problem.

I use Google Apps OAuth for everything possible. This makes it pretty easy. You can also use SAML or any other SSO mechanism.

You should absolutely create new users when that is an option. Then, when someone leaves, it's easy to remove access without having to worry about things.

We use Lastpass to store all company passwords, and there are some shares done through it as well, but we don't share unless we have to.

If you are sharing an account with someone you should change the password for that account when they leave the company. If you're using something like Lastpass or similar this is easier to do than distributing the password another way.

How do you manage service specific permissions or groups if Google Apps is the auth provider?

Thycotic Secret server.[0]

it costs a lot and hasn't got functionality that couldn't be replicated by something open source, but for now it's that.

https://thycotic.com/products/secret-server/

Additionally:

A centrally managed daily rotating EFS (windows) volume which gets called by some powershell scripts when you want to look up the daily rotated admin password for a machine.

For when machines inevitably get knocked off the domain.

I think it's custom.

EDIT: it's from SANS.org

http://cyber-defense.sans.org/blog/2013/08/01/reset-local-ad...

I've been meaning, for the last couple years, to see if WebPasswordSafe (https://github.com/joshdrummond/webpasswordsafe) is a viable alternative to Secret Server for basic password storage functionality. Secret Server is reasonably nice, but a bit expensive for what it does.

When I was at Microsoft (DevDiv) we used AD, Forefront Identity Manager and an internal webapp called RAMweb to request to join controlled groups.

Them being Microsoft everything was in-house, though I did work on a team that did use some external resources - we traded passwords on an ad-hoc basis - "trusted" SharePoint pages or shared OneNote documents mostly - but this was never anything sensitive - think: accounts for sites like HackerRank or Lynda.

All of the startups I've been at use LastPass Enterprise, it works well.

Over the past ten years, I've seen a mix of LDAP, oAuth, passwords/SSH keys provisioned via orchestration tools, handing everything over to Google/Okta, and a binder full of root passwords and revocation keys.

My favorites (I'm in DevOps) include LDAP (as long as I don't have to maintain it personally) and orchestrated keys. Anything that makes it easy to go to one place, change a file, deploy, and someone is created or erased. I used to love 2FA, but the cost for misplacing your token for a day is too high. I haven't come up with a good alternative for that, though.

Before posting details of your company's security infrastructure here, ask your security team. Even if your infrastructure is technically perfect, there may be good reasons not to publicize its details.

And that is part if the reason why quite a number of us use pseudonyms:

A number of things are much safer to talk about once it isn't tied back to you, your family or your job in any obvious way.

Still agree that even on top of that, great care should be taken. (E.g. in case someone suddenly figure out and blurts out: "you must be x from y, aren't you?".)

We're a small company, but we have a lot of credentials that go around and we generally use Lastpass with a shared folder because it's supported on all platforms and free.

The only thing that bothers me (and can't be circumvented because of browserdesign) is the ability to get plaintext passwords from autocompleted forms. So if an employee wants to write down a password, there's nothing stopping them.

For access to our servers we require everyone to use SSH keys.

Meldium says they login remotely and then push the session to your browser (thus never revealing the password to the user) which sounds awesome in theory. I questioned them on this as I was able to save and view completed passwords, and it turns out they can only do it on about 10% of the services we use.

It annoys me so much, that I have to type my strong master pass to copy a damn password while it is lingering in plain text under these auto-filled dots. I'm just a little bit slower going through the browsers dev-tools without the master-pass.

I'm a system admin with about 40 passwords for work I need to remember so this year I've been using keeweb to keep a encrypted password file/searchable file. This year I started throwing the dreaded htaccess passwords around company only sites we control incase a old employee wants to jump back in until we can decide on a better system/ clean up how we do things.

We use ScaleFT, wrote a blog post about it here: https://www.jungledisk.com/blog/2016/07/13/behind-the-scene-...

We use RatticDB[1] which is a bit bare-bones (all the passwords are stored plaintext in a database) but does the job well enough (audit logs and credential groups with view/edit permissions).

It's less slick than commercial solutions like LastPass, but it's cheaper, and outside of proper authorisation models in web applications, which Facebook does well, it's all just copy-pasting passwords with more obfuscation.

You'd think that with so many engineers and so much love for brands Twitter could at least develop a proper concept of shared accounts, so I wouldn't have to venture into some deep dark settings menu to tweet from a company account instead of my personal one.

[1]: http://rattic.org/

Just some ideas:

Giving and managing access:

Have a very clear flowchart for when someone joins/leave a role what their privileges are. Ensure all departments sign up to this understanding.

Write in all department SOP/flowcharts/guidelines what to do when someone joins/leaves a role. This is your field, and by using their server they're running on it. So ensure they comply.

Audit:

Run monthly checks to the above have been done. Above should be approved by all dept. heads so just get whoever's in control approval of policy.

Use SOX404 style checking of access/other access. This just means taking samples of actions that could be taken, and checking them. 10-15% should be OK, depending on process/function.

These are all non-technical checks. Do check in with your compliance/legal function for their perspective of what's needed (and what to avoid keeping on disk - just disregard). They will likely have a lot of insights which are both technical and non-technical.

1Password for teams. Works great

This, for us too. We've moved on from LastPass for teams, 1password is far nicer.

Same here. There's a little bit of friction getting people to start and keep using this so I end up doing a fair amount of troubleshooting and reminding but overall I sleep better at night.

My company is also using 1Password for Teams, but found it surprisingly hard to get everyone to use the same naming scheme. At my company there are few existing 1Password users (personal license) who already maintain a large database of passwords. Then someone link team vault to their 1Password. Open the All Vaults view. OCD kicks in. Chaos ensues.

(e.g. "news.ycombinator.com/username" vs "Hacker News" vs "news.ycombinator.com" vs "Hacker News/username" vs etc.)

Azure Key Vault + some internal wrappers to integrate with our production and testing environments. We've got Just-in-Time access for our environments.

For the work network, different teams have different strategies, but generally a password safe. No shared creds.

The previous company I worked for used a combination of LDAP with Yubikeys for 2F. Was very secure and lots of services ran on an internal network which you had to use a VPN connection to access.

The company I'm at now is using LastPass shares (which the creds can easily be obtained as others here have mentioned) and for a lot of other services there is a default username and password based on your name.

I'm trying to change the security culture at my current company, but it's hard since a lot of "security" is more of a pain in the ass. Example, "Why do I need to put in this code if I already put in my username and password?"

Anybody here have experience with storing SSH public keys in LDAP or an alternative to syncing them across a range of servers?

I've had LDAP authentication to servers but it's always been with passwords.

I just recently had to solve this problem. I ended up using this Python package: https://github.com/jirutka/ssh-ldap-pubkey

Ah very nice! This is exactly what I'm looking for. Couple questions.

Does this support multiple sshPublicKey attributes or just one per user?

Any performance issues with constantly hitting LDAP?

Multiple keys seem to be fine.

I haven't seen performance issues, but it's a relatively small deployment in the scheme of things. There are also existing solutions for caching here. NSCD seems to be the go-to for caching LDAP query results directly. Alternately, you could cache credentials at the PAM level with pam-ccreds (Debian package name).

I've done this, and it works well.

I used NSS-PAM-LDAPD [1], and openssh-ldap-publickey [2] with OpenLDAP

If you want more info about the setup ping me (email in my profile).

1: https://arthurdejong.org/nss-pam-ldapd/ 2: https://github.com/AndriiGrytsenko/openssh-ldap-publickey

We, uhhhh, have a password tab on our team onenote?

It's fiiiiiiiiine... :)

We have an Excel workbook "hidden" on the file server.

About a week ago Excel decided to change ... (3 dots) in one of my password to one Unicode char. About an hour lost.

I've always worked at large companies with dedicated Infosec groups, so I'm surprised to hear security is so awfully lax at so many places. I thought maybe small companies would be using open source software and something like:

https://www.nitrokey.com/

although it clearly takes some know-how to get set up with something like this.

Shared spreadsheet in Google Docs. [sigh]

Super insecure, move to a encrypted team password tool ASAP.

Active Directory for internal + Azure Active Directory(With sync) for SAAS/Web products.

Internal auth strictly over LDAP or SSH keys, so we don't have to worry about that.

For external things we can't integrate into that we use a password manager (self-made). People get dropped into the categories they're supposed to access, and done.

Does anyone's company use Okta combined with Active Directory? It seems to work well for handling access to internal apps and integrates with a lot of the business focused SaaS, like Salesforce and our project management tool (Mingle atm).

My previous company used Okta, and it was a positive experience, though not overwhelmingly so. Web apps that don't integrate it are clunky, but there's no getting around that.

In my previous company which I left just in half day/ during lunch break.

They were having 'admin123@companyName' in case that password doesn't work they generate SHA and then put in the database field 'password'

Active Directory for everything internal

external stuff is managed via Teampass (on premise shared pw manager), which supports shared credentials and user-only credentials

although teampass is a bit wonky at times, it works pretty well for us

Web apps authenticate against a SAML IdP. The SAML IdP is among the very small list of things that is allowed to talk directly to Active Directory.

Access to specific applications is usually controlled by group membership.

Active Directory. Except for those services guarded by an owner who is always on vacation or too busy processing other requests when you need him. Typically shared passwords for nonprod.

We have all the passwords managed through Meldium, I find it pretty convenient although it's security is as strong as your google account.

It took me several reads to figure out how you could use a blog site to share passwords. And this is why you don't read HN before coffee.

In most cases I've seen: not well.

in github :(

We've just started using Google auth. So far seems to work pretty well.

Keepass in shared folder

Do you know if there is a trick to granting different passwords to different users using Keepass/WebKeePass?

Surprised nobody has mentioned Hashicorp Vault.

Same. Are you using this to success? Do you mind if I asked you a few questions about it?

shared google drive folder with 1Password files

Azure AD

post-it notes

git-encrypt

I really like your approach, just ask people to share sensitive information on a public website. Anybody sharing their company credential storing policy here should be up for an early performance review by their managers, boss.

Here we start with not writing our policy on public websites.

I would argue the opposite. If your credential storing policy is so sensitive that it's not sharable, then it's probably not that secure in the first place. Besides that, how can I even possibly make use of that information when I don't even know the name of the company?

Not sharing the policy and not being secure are too different things. And ideally yes, you should be able to share it without increasing any risk, though real world tells us this is not the case. And not sharing this information is a more secure action then sharing it, even though there is no security through obscurity, there is certainly not security in telling people your credential storing policy on a public website.

It narrows down the interesting target(s) quite a bit for malicious script kiddy or hacker.

And regarding not knowing the company, you and quite a few others here, link quite a bit in your profile page, and leave a huge footprint on the internet with all sorts of information to use. So it's quite easy to make a profile of you, determine where you work, where you live, what you look like, and then knowing where your company stores the interesting bits... well its going to be a lot easier then going into something blind.

But what do i know, i'm not good at 'Hustling and exerting confidence.'

;)

You share the the policy with every employee that has a very real likelihood of becoming disgruntled in the future at some point. So arguably (and historically accurate, if I were to take a guess) you are sharing your policy with the most dangerous actors already. If sharing your policy has any affect on your security at all then you aren't secure and obviously you're sharing it with employees who are probably the highest risk of becoming malicious actors already so don't pretend not sharing it here does anything of consequence.

I tend to trust people i work with slightly more then random people on the internet i don't know. (almost tempted to explain how to avoid your 'issue' but then i'd be sharing what i warn against myself people should not share).

There probably also is a sticker on your frontdoor that says spare-key under the 2nd fake rock? Because any disgruntled visitor or family member you had might post that on Facebook anyway? Same principle applies, there is no benefit in having the public know this. Though when you did have a fight or disagreement with that visitor or family member you have chance to relocate the spare-key before that information is disclosed on their Facebook page.

So besides the negative effects i described, what good could come from posting your company (not yours to decide to share anyway) policy regarding login credentials?

Also i'd advice to check up on your contract what it says about sharing company policies and or secrets to 3rd parties (this site for example), before doing so. Because chances are somebodies' boss considers the credential policy as something that should be kept in-house and not public and made a point of this in his/her contract. No matter what you or i think about it, if they can argue you 'potentially hurt' the company you are screwed.

I tend to agree. Call me paranoid, but I can't help to see this type of questions (not this one in particular and not implying anything, I mean in general) as a bit of social engineering. Even when it is a genuinely innocent question, nothing stops third parties from benefiting from the information and making use of it.

This is why normal people consider IT a bunch of dicks.

Thank you very much for the public information disclosure about important infrastructure elements of your company!

Also very appreciated is the fact that many of you brilliant uberhaxors link to your company websites!

There is already a lot of valuable information available from the various "I am using an online service for all passwords"- discussions.

I guess the next step will be:

ASK HN - what are your top ten favorite passwords you use for your company infrastructure?

Best value generated by these threads: spot the brightest minds in the "Hacker" News community quickly! Do not forget to ask for the HN handle on every new business contact!

Either you're answering the wrong message, or your comment isn't useful at all. The question is VERY interesting and VERY OFTEN it's a critical point in many companies' infrastructure.

Agreed, but those who answer in detail should be mindful of if their handle or user description correlates or links to their business.

Reply on news.ycombinator.com