Clojure web security is worse than you think
hackworth.be
hackworth.be
I support a Clojure/ClojureScript web app in my day-to-day work. I've also written an OAuth2 workflow for friend (https://github.com/ddellacosta/friend-oauth2 ...pull requests welcome), one library Aaron Bedra spends a lot of time talking about. The story really is exactly how he describes--Clojure web security could be better. I suppose you could argue that, at least, the defaults are generally not bad...but that's because there aren't defaults in many cases.
I think the main issue is, ironically, the same thing that makes Clojure development so great: you end up composing a bunch of small libraries together, each of which do one thing well. But as Aaron points out, there are things that aren't taken care of and you can't assume they are done for you, and a lot of security is something that you don't technically need to get a web app working...
I wouldn't want to give up the approach of composing a bunch of libraries to build concise, easy-to-reason-about web apps in Clojure--I think it helps with security in and of itself--but the community could really benefit from having more libraries which nail these things down and help developers build web apps more safely, with less work.
That said, I don't think it's all bad: you can build high quality, secure web apps in Clojure, it just requires you to really pay attention to security. And we should be doing that anyways, right?
They struck me as both unrealistic considering the clojure community and a little too much "find the silver bullet" for my tastes. That said, I understand why a security focused person would want this and if my role was to be concerned about these things I would probably push in that direction as well. When taking a larger view however, this is a poor tradeoff IMO.
"I wouldn't want to give up the approach of composing a bunch of libraries to build concise, easy-to-reason-about web apps in Clojure--I think it helps with security in and of itself--but the community could really benefit from having more libraries which nail these things down and help developers build web apps more safely, with less work."
You nailed it, well said. I want a checklist and good libraries, not a framework that promises to handle things for you but quietly leaves a big hole open in that one part where you had to work around the framework to solve your problem (there is always at least one isn't there?).
You just can't avoid understanding these common attacks and how to mitigate them, at least at a surface level. You have to at least know where to apply the appropriate techniques, the implementation of that technique is left up to a well documented library written by experts.
"That said, I don't think it's all bad: you can build high quality, secure web apps in Clojure, it just requires you to really pay attention to security. And we should be doing that anyways, right?"
Not only should you be doing that, but if you aren't designing with security in mind then you are doing it wrong IMHO, "secure by default" framework or not. Either the people configuring the framework or the people writing the layers at the edges need to understand web security.
All that said, I understand the push in this direction. If you have a large team of junior devs then you want to limit their power. It's just not an approach that fits very well with Clojure. It's like Rich Hickey's attitude towards unbounded queues: there is no support for them, because you must be aware of this issue and hiding it causes bugs. There, now I've invoked Rich Hickey to justify my anti-framework bias ... so I must be right.
He has a point. Is one really that much better than another? Also, shouldn't it be possible to come up with a single library to do HTML escaping that has the suitably paranoid defaults, which is then used by all the templating languages? Is one templating language's code for that going to be so much revolutionarily better than another's?
"everyone use a (heavy, highly opinionated) framework", etc.
My understanding is that he pointed that out as a solution other environments have taken (Rails) but that there was also value in the composable way of doing things in Clojure. His actual position, I understood as, "there must be some way of unifying this for Clojure."
It would be business suicide to demand all your web designers learn clojure edn syntax in order to write out HTML in hiccup. They have work to do and they are experts with their existing tools, including quite complex IDEs.
If the html is being generated programmatically by clojure devs then hiccup type formats are extremely efficient. String building techniques suck worse for HTML than they do for SQL.
By far the most common types of template languages are the placeholder type. People love them. I hate them, moustache/ctemplate style templates are an excellent attempt to remove logic from them but why not just go all the way towards an enlive style if you recognize this benefit? I'm not the right person to defend them but the general argument is that designers can learn the crippled DSL easily and they can implement the simple parts of the template logic themselves. Moustache style templates are also useful if you are templating something like plain text that doesn't have structure.
And yes, I absolutely think that the single library to do HTML escaping with extremely paranoid defaults is the right solution. But a security person is going to be nervous that developers will not use it every place they should and unless I'm remembering incorrectly the request was for this to be integrated into the templating library to remove that error path. I'm not even sure how that could work properly outside of the placeholder paradigm (it has been done well in Haskell using types to track escaped/unescaped segments if memory serves).
In regards to frameworks: I think you are right about this position, I just really very much disagree that there "must be some way of unifying this in clojure" outside of an opinionated framework. The problem is the lack of ability to enforce a specific dataflow / data transformations that can be audited for security and generally applied. Enforcing a specific dataflow is the definition of a framework IMHO.
I fully agree with his push for more and better web security related libraries (I'm tired of doing these things myself but don't have the security expertise to create general solutions that anyone should trust) but some of it struck me as "if we just limit the expressive power of clojure for web development we can mitigate a large swath of security issues". He's absolutely right, but I have found that tradeoff to be a very poor one to choose outside of the simplest of CRUD apps.
Right, that's done in Hamlet (which was developed for Yesod, but is its own library).
As a community we should highlight libraries with a focus on security as well. I think key is focusing on making this a community consideration early and not falling into the same pit of issues as other popular languages/frameworks which shall remain nameless for the purpose of this comment.
Even if using a web framework which supports security out of the box, don't take it for granted either. For example: that's a common error developers make when using Spring Security. It's powerful and full-featured, and developers get security and everything in a breeze, but when doing things on the fly misconfiguration happens and vulnerabilities are out there in the open.
After working so much in Django, I've been interested in smaller, leaner solutions, but then, as my feature list keeps growing, I find I miss all the bells and whistles provided by larger frameworks. I did not think of the security implication before reading this article.
Yes, and they do. And there's no "lesser extent"; if I can't set up a fresh node app with the correct defaults for all of this stuff - if it's not safe out of the box, and I don't have to go out of my way to pick crappy settings - then it's just as vulnerable.
People made a lot of fun out of the Rails vulns that hit last year but they were the visible growing pains of having to mature a framework. It's not about the individual bits, it's about the ecosystem at large.
Devil's Advocacy: How about a front-facing Web Server/WebApp Firewall/language hosting environment? Build something language neutral that can be trivially set up with no more than 3 command-line options and the name of the language/web-framework and you've got the secure defaults out of the box?
Still, if there was a standard for middleware in general you probably could at least get all the headers right and provide some sort of safe, general CSRF, session, and authentication (not authorization) framework that way. Unfortunately, the easiest places to stick a generalized middleware, which is probably an nginx module, also tend to lack any sort of backend storage, which is a problem.
Whether or not you could sell anyone on the benefits of that would be an interesting question. Getting any sort of XSS protection would involve essentially putting a full Web Application Firewall in front, for instance, which would be infeasible with any reasonable effort, and I'm not sure anyone's going to look twice at a "web security platform" that does little to nothing (effective) against XSS.
Another talk from the conference I'm glad was presented is "How Clojure Works"[1]. Towards the end, he shows the negative effects on tooling caused by Clojure's dynamic + hosted nature. Clojure's stacktraces are a nightmare. And the painful startup time kind of negates the excitement a newcomer might have towards REPL-oriented development.
Hopefully it won't be too long until things improve.
[1]: https://www.youtube.com/watch?v=8NUI07y1SlQ&list=PLZdCLR02gr...
But you just need to start up once and then work all day in the REPL. Unless you mean that the JVM startup means that Clojure is not a good choice for writing utility scripts and such.
EDIT: As pointed out to me below, and in the excellent talk linked above, it's much more the Clojure bootstrapping time, which almost dwarfs the JVM startup time.
But even more surprising: You could take the default VisualWorks, tell it to skip the herald sound and the splash screen and shut a few other things off, and it would load its 12MB default image faster than Perl 5 could start up with a 768k memory footprint.
I would hazard a guess that some of this reflects on the Clojure community. (Not that this is somehow inherent or uncorrectable.)
Which is a shame, because I'd really like to be able to use Common Lisp for web development. But as much as I love the language, you have to be reasonable.
Or alternatively, you just need to be careful with keeping things simple and straightforward. IMHO frameworks only hide problems somewhere else beneath layers of complexity. Assuming a fixed bug-rate, the more code there is, the more bugs that will be present (and whether they get discovered is another issue.)
You still have to pay attention to XSS when writing your app -- there's no way around that -- but I don't think there are a lot of holes in the framework itself, and it makes it relatively easy to write a secure app.
I can't offer you a guarantee, of course, but I can tell you that I'm less nervous about Weblocks than I would be about Rails.
EDITED to add: Weblocks does have a pretty substantial learning curve, but once you know how to use it, it can make for very high productivity.
Sure it's more work, but in my mind, it's worth it.
This is a standard developer response - I recall when Sinatra was a fresh kid on the block, and the same arguments people used against Rails - but just because something is more fun doesn't mean you're going to get all these details right.
The attack surface in modern web apps can be such a large search space that the majority of people will never get it right. The point behind frameworks is that you can engage in efficiencies in scale when it comes to sensible defaults; even the most well seasoned and security paranoid developer can drown in the cornucopia of details that all have to be done right.
vs. a what? I've worn many hats in my day, the least of which has been a developer hat.
There's equal risk in relying on a code base incorporating features with underlying concepts that you or your team does not understand.
I'm all for having a framework, where you can review the code and be comfortable with what you're relying on, but in the absence of having a good one, well then build one. The benefits Clojure brings to the table more than make up for the work in either case.
There's just no excuse for not knowing the concepts and not knowing if your product needs them.
No one's saying "stop using clojure". The argument here is "build a clojure framework lots of people use".
>There's equal risk in relying on a code base incorporating features with underlying concepts that you or your team does not understand.
You do this all the time. I mean, I understand on a high level what my computer does but I can't describe to you the exact tree balancing algorithm my filesystem uses, how my CPU performs cache invalidation and branch prediction or even how exactly the JVM works.
Computers are too complex to not rely on underlying concepts you or your team doesn't understand.
The point is, within just the context of a web developer developing a web application there is too much stuff to know to always get it right. We need to reduce the number of things people have to think about, or else we'll always miss something.
The argument I am addressing is that people are deploying products with poor security because a great, all encompassing, framework does not exist. There's no excuse for doing that.
This is not about having to understand cryptography, it's about knowing when and where you need to use it and making sure it's included when you do. The issues outlined in the video are trivial.
Setting up a scenario where everyone has to get everything right all of the time is setting everyone up for a lot of heartbreak.
Having bad password management and opening everyone up to CSRF and XSS is not "trivial". They're gotchas, and looking out for gotchas is a waste of everyone's time.
As I said, I'm all for having a great framework, but I would not excuse the mistakes being made here, and I'm simply saying that I believe that my products and my knowledge level benefited by not having one.
> Having bad password management and opening everyone up to CSRF and XSS is not "trivial".
I'm not suggesting everyone should need to be aware of these issues, I'm saying someone on your team should be. And if you're a team of one, be prepared to make mistakes (which will not be limited to the issues outlined here).
Well, you should! These are mind numbing details that are best taken care of by other people. I am too concerned with building a good product to want to spend too much time going over mind numbing details.
The argument from "I become a more well rounded developer" is valid, but these should be things you learn once and then stop worrying about forever. There's a lot of magic in Rails, but it all goes away given enough time. Better abstractions are our bread and butter and we'll get nothing done if we continue to shy away from them.
What I mean to say is the easier it becomes for everyone to get everything right all of the time the more likely it is to occur.
Expecting everyone to individually know and understand every single gotcha is unrealistic compared to distributing the work load across a framework.
And the more time passes, the more problems with Rails are being discovered that have to be altered in a non-BC way.
A library makes no assumptions. It's just there and you decide how to use it. In the right hands it's more powerful than a rigid framework.
A framework that tries to teach you how to do things is by definition a pair of training wheels. Not the best setup for competing in Tour De France.
The solution to incompetent developers isn't a bigger, more rigid framework. It's education and training.
In aerospace, it's understood that better training can be part of the solution, but often better instruments and interfaces and well designed automated tools can be lifesaving as well.
A framework in aerospace would be more akin to a pre-built airplane with a bunch of holes where you put seats, in-flight entertainment consoles and other auxiliary non-critical equipment (because remember, airplanes are scary and you were told you can't do it yourself). Then you paint it and stick your logo on it.
Which is all fine, if it does the job for you, but you're not designing the airplane, anymore than microwaving TV lunch is you cooking. And just like warming up TV lunch doesn't make your job title a "chef", I don't believe people relying on frameworks to do the hard thinking for them deserve the title "architect" or even "senior developer". Which, again, is all fine, unless you or your boss are kidding yourselves about it.
And then one day the military, say, realize they now have a bunch of useless "military jets" built on top of a "passenger airplane framework". And NASA's space mission based on that framework isn't going too well, either. Whoops. Turns out frameworks aren't one-size-fits-all.
In a nutshell, apps have to be architected by competent architects, using simple, modular, does-one-thing-well libraries. These people know how to achieve scalability and security taking the particular needs of the app at hand.
The presenter in that video said that making assumptions is dangerous. There's no greater assumption than the fact a framework is right for your app, without understanding what this framework does under the hood. And there's no type of component making more assumptions for your app than a framework. Rails calls it "opinionated". I call it: you're the framework's slave, not the other way around.
There's no way around it. I know when Rails came out the hype was so huge, that everyone believed they and their dog can write big complex web apps. Nope. The only result of the framework mentality is a bunch of people making beginner mistakes and getting hacked all over the web.
The He 111 started out as a civilian plane, then became a bomber. But to be fair, it was always a bomber masquerading as a civilian plane.
http://en.wikipedia.org/wiki/Heinkel_He_111
Which is all fine, if it does the job for you, but you're not designing the airplane, anymore than microwaving TV lunch is cooking.
I don't think this is a very good analogy at all, unless it comes to something like App Engine or Heroku, and I'm not sure even then.
I know when Rails came out the hype was so huge, that everyone believed they and their dog can write big complex web apps. Nope. The only result of that is a bunch of people making beginner mistakes and getting "pwned" all over the web.
The smart move would've been to see this coming, then intervene, thus cementing that big huge groundswell of mindshare with better informed members of the community as a result. The thing is, we programmers are more apt to act like old-time barnstormers and solemnly shake our heads at the pilot who stalled and crashed, than to invent stick shakers and automated stall warnings.
Web frameworks should handle input (routing, request fields, url query) and output (templates). That's it.
We live in an age when having a complementary iOS/Android app for your site is not the exception anymore. Without a service layer you'll have coupled your services with your web pages. Connecting your server with your apps will be quite painful. You don't want that.
Done right, your service layers keeps your code simple, secure, and easy to debug, even if it's web-only. I'd argue that anything else is just due to lack of experience.
A service layer also makes the work on a project more parallelizable in terms of number of developers that can work together on it, each focusing on their part of the puzzle, without breaking each other's code all the time.
And moving the problem actually helps, when the original place was the wrong place to solve the problem.
Security is one problem when you write queries all over the place. The other two problems are data integrity, and maintenance.
In a service layer you have exactly three concerns:
1) Validate abstract input (and permissions of the caller).
2) Perform the transaction.
3) Return abstract output (and/or errors).
No routers, no controllers, no views, no templates, no CSRF, no XSS, no HTML escaping, no GET, no POST, no nothing.
Just input, transaction, output. Pure data. Pure business logic.
And suddenly things that seemed hard to get right, or things you had to repeat all over the place in your code, get done simply, and just once. And all your web code calls the service layer.
All your iOS and Android app code - also calls the same service layer.
And they're both secure because you need to get the service secure just once - then all interfaces (web, mobile, desktop, API) use the same service.
And when you don't isolate the service, you'll be running SQL all over the place, and use every framework's souped up solutions to avoid SQLi.
Which one do you feel is better?
SOA is great for scalability, but not really that much of a boon for security (especially since you now have a problem of authentication between your frontend service and your backend service which is often ignored since it's behind the firewall and nobody is going to be snooping.) (Other than the NSA, GHCQ, and China.)
But in a service layer you need to get it right once. And therefore if you don't, you have to fix it just once.
And coupling your business logic with your frontend layer suggests spaghetti code and violates DRY, because you typically have many frontends, but one app state.
Security is about focus. If the service coders can focus on the service being secure & fast, frontend guys can focus on the frontend being usable.
Otherwise you're asking everyone to think about everything, and human attention span, memory and skill sets are limited, and this does affect security.
For example, the CSRF section could briefly explain what cross-site request forgery is, `<img src="http://example.com/logout">`, and recommend some approaches and good practices.
Code examples could be provided for various languages/frameworks each in its own tab. The Clojure tab could demonstrate https://github.com/weavejester/ring-anti-forgery and a `anti-forgery-form-to` macro that wraps `hiccup.form/form-to` and inserts `(ring.util.anti-forgery/anti-forgery-field)` into the form.
The XSS section could remind us to ensure that our templates are escaping user input. The Clojure tab could chart the safe vs unsafe templating libs. (Hiccup is unsafe)
The Sessions section could remind us to sign our cookies so that someone can't just change a cookie from {:user-id 42} to {:user-id 1}. And that our never-expiring authentication cookie means that anyone with that cookie can log in as that user forever. It could explain an overview of the main solutions and their drawbacks like how to back sessions by a database or set an expiration within the signed cookie.
Just like convos about crypto, discussions about security too frequently turn into some abstinence-only pamphlet.
As opposed to a "Purity Test" as a checklist?
This is why the solution isn't relying on your framework, but on your competence.
> and session cookies don't default to http-only flags (allowing XSS attacks to steal them).
Can someone explain that to me?
This brings to mind an incident, about which I'll say: While I do find that the community is friendly and a source of satisfying discussion in person, the Clojure community online is notably less beginner friendly than other language communities. (tl;dr - Attempts to "help" newbs that are really opportunities to be a language weenie considered harmful.)
I find some parallels with the Smalltalk community, which had a reputation for an "ivory tower" attitude and a "Not Invented Here" snobbishness. When I was a beginner in Smalltalk, a coworker of mine had written a nice but arbitrary little DSL specification language using Array literals and symbols. So I made some sort of comment about the syntax she'd created, and which prompted a knee-jerk reaction about how Smalltalk had a small non-ALGOL syntax -- as if I were totally unfamiliar with Smalltalk syntax.
I've had experiences like this asking about the arbitrary function definition-like syntax of Compojure web application routes. Those look somewhat like function definitions, but they're really macros. I had tried for some time to Google the answer, but no one had ever posted about the exact code to do what I wanted. (Which wasn't at all abstruse. Something like getting access to the entire request map while modifying the returned session.) Well, it turned out that all I needed to know was to not refer to the request as
[request]
request
Just what kind of community treats with newcomers by assuming that the newcomers are lying and then telling them to go read Lisp macros? Also, this is only the most easily related situation I've encountered while interacting with the Clojure community. There have been others.
No non-algol patterned language has ever gained more than a niche acceptance. If Clojure doesn't want to commit long term mindshare suicide, some attention needs to be paid to avoiding inadvertent hostility to newcomers. (This includes unintentionally being a bad security-defaults minefield.) Smalltalkers had the same "read the source" knee-jerk. In the end, it didn't work out so well. Care needs to be taken to understand exactly what the newbs are asking, because they know the least about how to formulate the question. Also, if you find yourself running a simplistic "pattern match" head-daemon eagerly looking for opportunities to bedazzle the newb with your wonderful, superior, and different way of doing things, just stop. You are giving your community a "weenie" reputation and your mental energies are better used elsewhere.
They seem generally pretty helpful to newbies, from my own selfish perspective to a fault, since I have to skip past the dozens of "how do I add to arrays" with clojurebot responses.
It might be in the way you're asking.
Lots of them seem to be stuck in "quick pattern match" mode, so in the other occasions they were "pattern matching" on my asking about a much simpler scenario than I was actually asking about -- then taking my trying to inform them their assumptions were wrong to conclude something like "he doesn't understand threads."
It might be in the way you're asking.
I say as much in my comment above -- I don't know enough about how to ask the questions in the most recognizable way. However, there seems to be an eagerness to throw everyone who doesn't otherwise match in the "total newb" bin, and once that happens, an inability to parse facts presented that should indicate that their initial mental models were wrong. ("You can't do that with thousands of threads."/"its dozens of threads" -- (EDIT: and actually, it can be as few threads as one wants.))
I posit this: There are other people like me who have asked non-typical questions, were treated in this way, then just turned their backs on Clojure. I just happen to have a very strong allergy to counterfactuals and make noise in this situation.
Macros (and DSLs) are more like their own little languages than they are like simple functions. To understand a macro on a fundamental level, you have to feel the need which suggested its creation. And the simplest way to do that, I think, is to read the macro's definition, to see what the code would look like without the macro there to clean it up.
This means, though, that to the extent that a library is definted in terms of macros, you have to meditate upon that library to understand not just how it was written, but why. This is the barrier macros create: without them, you can wave around other people's APIs and code in a cargo-cult fashion, and you'll probably get what you want. Macros prohibit cargo-culting a solution together: you actually have to understand the goals the library was built to serve.
All the OP needed is an explanation of the syntax of a Compojure macro. If someone is asking about the syntax of a specific macro, then "you don't understand macros" is not the correct response.
And that sort of thing doesn't work anyway for 90% of people.
Anyway, macros are easy to understand. They're hard to grok, but easy to understand. Generally you grok things by using them.
Me: What's the preferred Clojure solution for this highly specific performance issue that I've tracked down?
IRC: Did you benchmark your code? Have you tried using functions?
Me: groan
Edit: Perhaps there are some people present who want to defend the Clojure IRC channel... I'm being 100% honest with my experiences, only slightly paraphrasing things for humor value. The above conversation actually happened, in a more verbose form.
He mixes password hashes and MAC (message authentication codes), saying we should HMAC our password hashes. There's little to gain from that:
- Password hashes exist to hide passwords. The attack is to discover the original password by brute force, dictionary, rainbow tables etc. The protection against this is irreducibly slow hash algorithms and unique public salt per hash.
- MAC exists to authenticate message origin. Unlike passwords, the message is often public. The attack is therefore not to discover the message, but to manipulate the message, yet make it appear from the same origin as the original message. The protection against this is the HMAC algorithm with a secret key that only the original message author has.
HMAC-ing your password hash won't stop, or even significantly slow down any of the attacks performed against password hashes, so it feels cargo cultish to claim it's just magically more secure by using it. It's even more WTF-y to claim a password hash library should offer HMAC, or it's insecure.
Once a system is compromised, and the attacker doesn't care to recover the original password, it's far more trivial to just replace the hash or just directly read the data protected by the login directly in the DB, rather than play out a MAC attack on a pass hash. It makes no sense.
HTML escaping. You either escape for HTML or you don't. There's no sane way to escape some of it. His idea that we should be able to escape some tags, but skip (or "strip") others is a huge vector of attack as HTML is notoriously filled with edge cases that your selective escaping/stripping engine won't have. It's not trivial to parse what's a tag and what isn't a tag to a browser, trust me. It only looks trivial if you're ignorant.
If you want to allow some tags, and disable others (say, allow bold/italic on forums, but nothing else) you should go through a strict DSL allowing only those types of formatting, which are then converted to HTML. Just like HackerNews does it - I type star-word-star and I get "word" in italic. That's a DSL. That DSL could be made to look like HTML, but it won't be HTML. It's fully parsed and rebuilt from the DSL syntax tree. So it can be made safe. Selective HTML escaping without those crucial steps is almost impossible to make safe by comparison.
His rant about "there are too many templating engines - we should stop" - yeah, good luck saying we should stop and this working out.
Likewise about databases. "Let's just stop". B.S. The thing he's missing is that different DB engines have different strengths and weaknesses. That's one major reason there are so many, and why many get used in the same project. PgSQL isn't SQLite isn't MongoDB isn't Redis.
By enforcing one type of database for multiple separate project components, you don't improve security, you just cripple architecture and performance.
(def a-password (-> pass
(hmac server-stored-key)
(scrypt-kdf work-factor))
You could replace HMAC with encrypt with AES (or scrypt for that matter) to get the same practical effect.
Edit: clarification
Encryption also does nothing here, because it's reversible.
Your encryption key is somewhere on your server, so whatever you encrypted (either the plain text password, or the hash) can be decrypted once, and then if it's a hash you just attack the bare hash as usual (brute force, dictionary etc.).
Instead of encryption you can just store part of the salt outside the DB, but with a good hash this is usually not needed and just pointless obfuscation.
Public access to a SQL database. Input isn't properly sanitized while building some complicated query. $badnick executes a SQL injection. Luckily, the database user from the public site only has SELECT privileges (yay SOA or something). This is an extremely common enterprise situation.
The attacker can dump data (like password hashes), but in an HMAC'd password setup, cannot actually gain access to modify or login as a user via brute-forcing. They cannot pivot the user account to access to another system. The attacker must find a different hole to gain access to the app/login server.
1) The SQL database is not public (any more than it's common to have public anonymous FTP to your server's app deployment). It listens on a specific interface, specific port, from specific IPs with a specific user and password (which is not "root"/"").
2) Input isn't properly sanitized? Hello? If that's "common" then fix that first. It's not 1998 anymore, no one has an excuse for that level of ignorance. First of all "sanitize" is an incorrect term for what needs to be done. You either escape an SQL literal, or you pass it as a parameter to a prepared statement. You do this to all data, all DB layers provide escaping, and it's absolutely trivial to do. And for prepared statements, there is no way to do it wrong, because it's not you who does it. Data and query in prepared statements are separated at the protocol itself and injection is impossible, you can't possibly mess it up. All of this is "using databases 101". Heck, your app will break even with innocent data containing, say, a quote, if you fail those.
3) I mentioned twice already, there's public salt for hashing the password, and there's irreducibly slow hash to be used to protect against a weak password (like Blowfish). This is what breaks brute-force attacks, not HMAC.
4) Let's assume your exact scenario is indeed terrifyingly common. Still, HMAC does nothing at all here, that keeping part of the salt off-the-DB won't do. HMAC doesn't mean "when you have a key outside the DB". You don't need HMAC to keep part of the salt outside the DB. Just... keep part of the salt outside the DB, without HMAC.
But if you feel the need to obfuscate your code like this, it means you're working in hell and some people don't deserve their jobs, and better fix that, before you start thinking about compensating for it at the wrong place of the app.
1) The SQL database is not public (any more than it's common to have public anonymous FTP to your source code). It listens to a specific interface, from specific IPs with a specific user and password (which is not "root"/"").
Yes, mostly true, but in the common case passwords are shared between users, and in corporate apps there are some widely shared "admin" accounts that are user accounts with some added capabilities, or some such bad practice.
you pass it as a parameter to a prepared statement. You do this to all data, all DB layers provide escaping, and it's absolutely trivial to do. And for prepared statements, there is no way to do it wrong
Yeah, so what exactly is wrong with SQL Korma? I thought that would be one succinct and abstracted way of making sure all requests are through prepared statements.
I.e. User <-> App <-> Database
Input is simply not properly sanitized in the real world because mistakes happen. This isn't by design. This is a source of remote execution. The remote execution only yields read access since the app only needs read access (and thus the fewest privileges possible are read access). The attacker can read password hashes, but cannot update them.
As I mentioned before, HMAC is just an example of keeping all information necessary to login to the site out of the stored hash. Any non-constant value per password will do (feel free to maintain a lookup table of unique lists to active passwords in your app if you feel you must, I'll be HMAC'ing the user's input password against a constant secret key before sending to KDF). The idea is that an attacker who can read your database (but not the application code) cannot gain access as a user without first also compromising the application code.
And once again, you don't need HMAC to store salt outside the hash. You just don't. HMAC doesn't dictate where you store your salt. You're bundling these two things together as if they're inseparable, but they're two completely separate things.
I'm not up to the play with HMAC so can't confidently comment on that, but the point being made is that there is lots of SQL injection out there. Keeping the salt out of the DB seems like a not-insane way to help mitigate a compromise.
Let me ask, does this hypothetical company have someone in charge of architecture, or everyone just codes randomly entirely on their own with no rhyme or reason until it works? Because that's what you're describing. Anarchy & chaos, with no one in charge.
The answer to the above hypothetical situation is that the PHP developer will code to the service layer, not to the database. In fact, that precise setup is very common in the projects I do: services (say in Java) consumed by delivery mechanisms (say PHP web sites).
Multiple independent apps reaching into the database directly is not only recipe for a security disaster, but a recipe for data integrity and maintenance disaster.
Here I explained service layers, so I won't repeat myself: https://news.ycombinator.com/item?id=7476725
(ns passwords
(:require [pandect.core :refer [sha256-hmac]]
[crypto.password.scrypt :as scrypt]
[environ.core :refer [env]]))
(def SECRET-KEY (if-let [key (env :secret-key)]
key
(throw "Set your SECRET_KEY!!!")))
(defn encrypt-password
[pass]
(-> pass (sha256-hmac SECRET-KEY) scrypt/encrypt))