Millions of accounts compromised because there is no specialised user database
fourlightyears.blogspot.com
fourlightyears.blogspot.com
There are dozens of directory server options, and probably a few dozen secure authorization solutions, from Kerberos and SAML to proprietary solutions like SiteMinder. Together, these solutions give you EIAM capability that does what you described.
I wouldn't characterize the magical detection of uncommon access patterns as a minimalist feature (it can be done, but not out of the box in a meaningful way), but other than that, all of the stuff you mentioned is there.
Typically, you use LDAP as a directory store, and a dedicated authentication protocol like Kerberos or SAML for authentication. You can also use LDAP for auth if desired.
If you want to go deep on identity management, the NIST documents are pretty much the canonical source of information. http://csrc.nist.gov/projects/iden_ac.html
In any case, don't reinvent the wheel. Just because bozos implement user auth in a MySQL table doesn't mean solutions don't exist. It just means that they are bozos. It's one of the reasons I disapprove use of SaaS apps that don't support federated identity at work.
I get the impression that OpenLDAP is a big beast that does much, much more.
I'm wanting a fast motorbike and you're pointing at a 16 wheel heavy hauler saying "see, same thing, the hauler has at least two wheels and an engine".
You also said:
"Another problem is that developers roll their own user and password management systems and get things like salting and hashing wrong, making the data vulnerable."
You're asking for a distributed, secure system for processing user data and credentials. The most common systems in production globally for doing this (Microsoft Active Directory) are based on technologies developed in the 80's and only being supplanted in recent years. There's a reason for that, and if it were a trivial problem, we wouldn't have pervasive solutions using 1980s RPC in 2015 still running most businesses!
If you want a secure system you shouldn't be prioritising a "minimal solution". As the previous poster said, don't reinvent the wheel.
I'm not even sure about a meaningful way. For sake of portability the DB will need to have some way to dump everything. And if a server admin can do that, then someone who can get the same access as a server admin can do it, and we're back to square one.
It supports a lot of backends (SQL and NoSQL alike) (openLDAP has nice proxy features)
It also supports role based authentication.
But at the opposite of actual NoSQL data bases, even though it has no structures, it is strongly typed and secure.
You map your NoSQL structures to structured backend through a mapping.
It is awesome but most devs hates it, because they are (at my opinions) ignorants. And developers hates it because they are lazy.
There is a need for something secure by default that is easy to use, easy to set up, easy to deploy, etc. Where this will be difficult is https://xkcd.com/927/.
It's why everyone uses JSON and not ASN.1. Why people will save a single number in a file rather than standing up Oracle database. Or why they still use random invented rot-level "encryption" for messages rather than pgp.
LDAP doesn't expose operations like "create new user", "verify password", "deactivate user", "set encryption method to ...". It's a database where you "bind" your user description against your hierarchy of objects/attributes - explain that to a guy who just wants to get his new app idea working ASAP... It's got its own query language too. You can even do LDAP query injections - just like in SQL!
Like you said - LDAP is a beast. It's great and I'd use it as soon as number of users goes over 10k, or employees on different levels over 10. But otherwise, no way. A new component with its own security issues is just not worth it.
And that's just when you're using the recommended connectors (ODBC/ADO.NET). FSM help you if you're using JDBC and the vendor of the $20k+ product hasn't tested against Server 2012.
I wonder if there's any money in providing an LDAP/AD -> REST server.
http://stackoverflow.com/questions/2453769/active-directory-... might be an idea, too.
Unless you escape your "(cn=$username)" properly, the users can do a lot of fun things.
More importantly, an LDAP search filter only goes with an LDAP search request, which is read-only. You cannot use silly parser games like this to destroy a database, like you can in SQL with Little Bobby Tables. https://xkcd.com/327/
Playing games with LDAP search filters can't return you more information than the app was originally authorized to see. It most often won't even return you more than a single record, since most apps that make LDAP queries and expect only a single entry to be returned won't even look at any subsequent entries if they happen to be in the reply.
Canonical LDAP application code to search for a user record, for authentication/authorization purposes, sets the LDAP search sizelimit to 1. Getting more than 1 matching result is an error.
But it does exist. Saying that LDAP auth[nz] code should set sizelimit to 1 is like saying SQL auth[nz] code should set LIMIT 1. Correct and defensive code does that. But there's still code out there which doesn't and is vulnerable.
Then again, I don't think you understood the second example. It's not for auth, but rather user search - which is the probably the second most common corporate ldap usage.
As for LDAP injection being nonsense, how about searching some CVEs:
- LDAP injection and resulting STORED Cross Site Scripting in Events Planner - SmarterMail 7.x (7.2.3925) - https://www.exploit-db.com/exploits/15189/
- Graylog2 before 0.92 allows remote attackers to bypass LDAP authentication via crafted wildcards. - CVE-2014-9217
- PowerDNS before 2.9.18, when running with an LDAP backend, does not properly escape LDAP queries, which allows remote attackers to cause a denial of service (failure to answer ldap questions) and possibly conduct an LDAP injection attack. - CVE-2005-2301
And many many more...
I concede the point that there's poorly written code out there that is vulnerable. Still, to lump this into the label of "injection attack" along the lines of SQL injection attacks is grossly misleading. An SQL injection attack can cause destruction of data, invocation of arbitrary commands on the backend, and innumerable kinds of other mischief. A misparsed LDAP filter can do none of these things.
I can find no clear answer after a few minutes of searching.
So, practically speaking - does it matter?
[0] And equivalent for each country with >70M people
http://symas.com/docs/2014-LOADays-jkozyra.pdf
80% of the US population that owns cellphones are OpenLDAP users, whether they realize it or not. 90% of the US population that subscribes to cable TV (Comcast, Time-Warner, etc.) as well. Nothing else actually works for hundreds of millions of users. These companies know, they've tried a lot of "solutions" already.
My own personal experience with LDAP in general (and some with OpenLDAP in particular) dates back to 2000 or so -- and at some point I just gave up on the complex and unreliable beast that it was (for whatever reason, BDB would occasionally lock up or get corrupted) - but I know a lot has changed since, LMDB is amazing -- I guess it is time for another look.
The core proposition of this article is that if we make it really easy to store user data correctly, people will do it. This proposition is simply not borne out by the data. Many parts of user security are easy. Take password storage, for example: it's so easy a caveman could do it. Every major languge has a library that takes a password and a salt and a number of iterations and runs it through pbkdf2 or scrypt. Many have these functions builtin. But do people do it? No. It seems like once a month some company exposes X-thousand passwords because they stored it in plaintext or MD5. And I bet a significant portion of HN readers think SHA1 is a valid way of storing passwords, eager to sacrifice security as efficiently as possible. pbkdf2 and scrypt aren't hard to use, people are just idiots.
Making a specialized database just gives people another easy thing to not use.
Ultimately there's no reason to be giving your password to a service in the first place. You only should be giving them cryptographic proof that you have your password, not giving them your password. This is possible with zero-knowledge password proofs[1], but it would require browsers and other client-side software to implement a ZKPP algorithm. This would force servers to either implement a ZKPP algorithm correctly or expose their incompetence to their users. Similar algorithms can be hypothesized for many other user data needs, but passwords would be an excellent start.
[1] https://en.wikipedia.org/wiki/Zero-knowledge_password_proof
It's not a problem of how it's being stored. It's a problem of what is being stored.
I would suggest that the world doesn't need a specialized database for this, it needs killer libraries that make all necessary operations dead simple with any backend. And they should be dead simple to find and install.
More fundamentally, though, I'm not sure this problem is entirely solvable. Even with all the libraries and informational websites in the world, if developers don't go out and look for that information (and find the right stuff), they may never know they're doing anything wrong.
P.S. Hello again, alter ego.
> It should be accessible only via its specialised API which is designed to constrain the ways that it is accessed.
Constrained in what ways? Especially in management / support roles you may need to do bulk actions against users (full listing, bulk changes, etc) so I'm not sure what constraints would be compared to a normal database.
> Its API should have password salting and hashing built in.
I disagree; I think password hashing should happen before the data hits the database.
> Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data.
Does someone have an implementation idea for this? Depending on the type of user data we're talking the data could be very small. So the throttling may have to be quite significant but even then what's to prevent the attacker from bypassing it by using multiple connections? Multiple threads that each request a subset? I don't think you can get around those and still have a performant system.
> It should encrypt data internally.
How? Does it encrypt using a passcode specified by the system administrator (which means it exists somewhere on the system or nearby system making it decrypt-able anyway)? Or does this mean encryption using the user's password thus separating the data? If so how do you handle the cases when administrators need to modify the user's information for support or other reasons?
Our needs are a little different as well, because each service may need to reach out to a third-party service and we are holding a bearer token for the third-party service on behalf of the user. We provide that to the services behind an auth middleware through a header.
(We had considered changing this from an Auth Service to a general User Service, but I am strongly leaning toward having a separate User Service and leaving Auth tightly constrained.)
1) Authentication data (login id?, password) 2) Personal data that is not used in an online manner (depends on the site of course) (full name, email address, etc.) 3) Data that shows up on the site
If this data is segregated in to separate silos with no 'read everything' apis exposed to the web servers, you might stand a chance of not leaking all of your users data when the web servers are compromised. This of course will not help you if the authentication servers are also compromised.
I was imagining a social-network use case while reading the post, so when a similar objection came to mind, the answer seemed obvious: make "users" and "accounts" separate things. The business-logic keeps around its own "users" table. The business logic can, and must, index that.
Each user then HAS A account; there's a 1:1 correspondence. The accounts are in a separate service (a AAA microservice, effectively) that exposes this minimal API where you can only talk about individual accounts to it.
How does a backoffice user-management dashboard do its job? Like the name suggests, by reading off the users table—and then doing O(N) queries to the accounts service to effect changes. These sort of one-off administrative operations are allowed to be costly like that.
LDAP can be painful to work with, even when using such "industry standards" like Active Directory
I think a specialised server (or even Postgres extension) could be useful but couldn't you set up the following?
* Create a stored proc or similar that does this: SELECT COUNT(PasswordHash) WHERE Username = @Username AND PasswordHash = @PasswordHash
* Create a user account on the database that only has EXEC permissions, not SELECT or anything else
* Restrict the user account to only be able to log in from the app servers (or whatever)
* Restrict the DB admin to only login via a jump box / VPN.
There are a ton of services you can federate with, it's easier for the user less passwords to remember.
If you really want users to authenticate natively. Take a look at one of the new-er players out there Storm Path. It's basically your IAM backend to-go. Don't write your own security if you don't have to :)
But depend on an NSA aware third party to store your client's credentials?
For example Symfony has FOSUserBundle [1] which provides nice support for most listed things. Django comes with its auth module [2]. There are more examples like that. They allow easy auth / user management without touching any of the underlying details. And once you actually want to go the direction of LDAP/kerberos/..., it's usually just a matter of switching the backend implementation for something that already exists.
[1] https://github.com/FriendsOfSymfony/FOSUserBundle [2] https://docs.djangoproject.com/en/1.8/topics/auth/default/
I tried to find decent hashing in OpenLDAP and Apache DS but didn't. Perhaps I didn't look hard enough. Do they throttle by default?
Who's gonna make LDAP webscale and hipster friendly?
Don't get me wrong, this won't solve everything, but in my opinion it's a pretty big step in the right direction.
1. s/specialized/separate/
2. Use SASL, or another authentication protocol to talk to your auth DB.
3. Profit!
Re: LDAP
This may not be as good as a specialized solution, but it's probably a step up from throwing all your data into the same MySQL DB. ;-)
It's not particularly hard, either, just different from what you may be used to.
From HN front page to solution within hours!
If it said "built with Golang" or "built with Rust" it would certainly hit the front page.
* Put the user database behind a microservice
* Use a different connection with different credentials and put appropriate permissions on the tables
* Store password hashes using bcrypt, scrypt or PBKDF2
Not to mention that if you write a new database, what you're actually writing is brand new exploitable bugs.
It's puzzling that you say this is "completely misguided" and describe exactly the same thing.
A Postgres database, for example, with correctly-limited user permissions set, and a good set of views and triggers and stored procedures, is exactly equivalent in power and flexibility to putting some blobs of Sinatra API-routing and model code in front of Postgres-used-as-a-dumb-object-store.
RDBMSes were always meant to do, on their own, exactly what microservices are doing now. That's why they're so much more complicated than just being dumb object stores: you can encode within a RDBMS schema the constraints of the model, and the valid transformations you can make to any given piece of data, to create an API exactly as narrow and high-level as a given user needs.
I think it was just some quirk of their use in mainframe contexts which lead to their auth protocols never being hardened to work on the public Internet, which meant people started to believe there was something inherently insecure about letting users talk to an RDBMS.
There's not; a database, a real database, is a persistent micoservice server.
And now we're all the way around to reinventing them on top of themselves, but this time they're three separate process-spaces, random custom languages are required, and they don't understand questions phrased as relational algebra.
thanks
It should not provide generalised database query functionality. YES - it only proivdes the subset of LDAP functionality necessary for authentication.
Its API should have password salting and hashing built in. NO - hashed but needs salting&scrypt.
Its API should throttle access with some sort of algorithm designed to prevent downloads of large quantities of user data. NO - todo.
It should encrypt data internally. YES (hashed not encrypted).
It should communicate only over encrypted connections. YES.
It should be distributed. YES.
It should not be run on any web server, should run "behind the scenes" and be accessible only via its API. YES.
It should include triggers and alerts based on uncommon access patterns or recognised nefarious access patterns. NO - todo.
It should have no other purpose. YES.
And It should encrypt data internally really gives no protection other than physical theft of the disk drive, as in if the portion of the application is compromised that can access the data, the key is available at that point, and the attacker has free run of that corner of the universe.
If anybody imagined that openssl had no bugs, they've obviously been proven wrong several times already, and I'm sure they will be proven wrong again.
The point of a single library (or few libraries) is (at least) twofold. It's much less likely to have defects, and when those defects that remain are found, answering the "does this affect me" question is easier and prompts quick action.